Blog | CoWhite Softwarehttp://django.cowhite.com/blog/2022-01-02T12:59:58+00:00BlogList Comprehensions vs For loops in python2022-01-02T12:59:58+00:00bhaskar/blog/author/bhaskar/http://django.cowhite.com/blog/lists-comprehensions-vs-for-loops-in-python/<p>Let us take a scenario where I want to get the list of even numbers from a given range of numbers.
We can get the list of even numbers using the for loop as shown below.</p>
<div class="codehilite"><pre>even_list = []
for i in range(10000):
if i%2 == 0:
even_list.append(i)
</pre></div>
<p>As shown in the above code, we are trying to create a list of even numbers from 0 to 9999
The same can be achieved using list comprehension in a one-line as shown below</p>
<div class="codehilite"><pre>even_list =[i for i in range(10000) if i%2==0]
</pre></div>
<p>So for creating a new list, when we compare the speed of execution in the case of for loop and list comprehension, list comprehension is two times faster than the for-loop.
The performance of the list comprehension decreases a bit with the increase in the complexity of the operation performed, but still, list comprehension manages to have better processing speed than the for-loop.
In the above example, the i%2 operation is a simple operation, so the speed of the list comprehension is 2 times better than the for-loop.
Let us take the below example</p>
<div class="codehilite"><pre>list1 = []
for i in range(10000):
list1.append((i*3)**4)
</pre></div>
<p>Here, we are multiplying every element with 3 and then the multiplication result is performed an exponentiation operation with 4 , and this is a complex operation
The equivalent list comprehension logic is shown below</p>
<div class="codehilite"><pre>list2 =[(i*3)**4 for i in range(10000)]
</pre></div>
<p>Even, in this case, the list comprehension has better performance than for loop, but it is not 2 times faster now, it is more like 1.2 times faster than for loop, so definitely the performance of the list comprehension decreases with the increase in the complexity of the operation.</p>Else clause after for and while loops in python2022-01-02T06:07:35+00:00bhaskar/blog/author/bhaskar/http://django.cowhite.com/blog/else-clause-after-for-and-while-loops-in-python/<p>Some of us are not aware of the else clause after the for and while loops in python.
Take this example</p>
<div class="codehilite"><pre>weekly_trends = {1000, 1100, 1500, 1250}
is_abnormal = False
for value in weekly_trends:
if value > 1400:
is_abnormal = True
break;
if not is_abnormal:
print("no abnormalities found")
</pre></div>
<p>The above program is maintaining a flag to check if any element has a value greater than 1400, if found, it is setting the flag is_abnormal to True, then executing the break statement. And, after the for loop, we are checking the flag and if it is not set to true, we are printing "no abnormalities found".</p>
<p>Instead of maintaining a separate flag, we can make use of the else clause after the for loop as shown below.</p>
<div class="codehilite"><pre>for value in weekly_trends:
if value > 1400:
break;
else:
print("no abnormalities found")
</pre></div>
<p>So, here, the control will come to else block, only when the break statement is not executed, and this would serve the same purpose without the additional flag logic.</p>Working with decorators in Flask2021-01-26T06:04:33+00:00Balaji P/blog/author/bljd369/http://django.cowhite.com/blog/working-with-decorators-in-flask-1/<h3>Decorators in Flask</h3>
<p>A decorator is a wonderful ingredient with which you can add more generic flavor to one or more functions. Using decorator, you can implement authentication, authorization, input validation, etc.</p>
<h3>Decorator usage for authentication</h3>
<div class="codehilite"><pre><span class="kn">from</span> <span class="nn">functools</span> <span class="kn">import</span> <span class="n">wraps</span>
<span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">g</span><span class="p">,</span> <span class="n">request</span><span class="p">,</span> <span class="n">redirect</span><span class="p">,</span> <span class="n">url_for</span>
<span class="k">def</span> <span class="nf">login_required</span><span class="p">(</span><span class="n">f</span><span class="p">):</span>
<span class="nd">@wraps</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">wrap</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span> <span class="n">kwargs</span><span class="p">):</span>
<span class="sd">'''if 'logged_in' in session:</span>
<span class="sd"> return f(*args, ** kwargs)'''</span>
<span class="n">authorization</span> <span class="o">=</span> <span class="bp">None</span>
<span class="k">if</span> <span class="s">"MyToken"</span> <span class="ow">in</span> <span class="n">request</span><span class="o">.</span><span class="n">headers</span><span class="p">:</span>
<span class="n">authorization</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">headers</span><span class="p">[</span><span class="s">'MyToken'</span><span class="p">]</span>
<span class="n">user_id</span> <span class="o">=</span> <span class="n">get_from_cache</span><span class="p">(</span><span class="s">"user-token-</span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">authorization</span><span class="p">)</span>
<span class="k">if</span> <span class="n">user_id</span><span class="p">:</span>
<span class="n">user_id</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">user_id</span><span class="p">)</span>
<span class="n">user_data</span> <span class="o">=</span> <span class="n">get_from_cache</span><span class="p">(</span><span class="s">"user-data-</span><span class="si">%s</span><span class="s">"</span> <span class="o">%</span> <span class="n">user_id</span><span class="p">)</span>
<span class="k">if</span> <span class="n">user_data</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span><span class="p">:</span>
<span class="n">user_data</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">user_data</span><span class="p">)</span>
<span class="n">set_user_details_in_session</span><span class="p">(</span><span class="n">user_data</span><span class="p">)</span>
<span class="n">request</span><span class="o">.</span><span class="n">user_id</span> <span class="o">=</span> <span class="n">user_id</span> <span class="c">## try to understand this line</span>
<span class="n">session</span><span class="p">[</span><span class="s">'user_id'</span><span class="p">]</span> <span class="o">=</span> <span class="n">user_id</span>
<span class="n">session</span><span class="p">[</span><span class="s">'email'</span><span class="p">]</span> <span class="o">=</span> <span class="n">user_data</span><span class="p">[</span><span class="s">'email'</span><span class="p">]</span>
<span class="k">return</span> <span class="n">f</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s">"error"</span><span class="p">:</span> <span class="s">"Invalid Token"</span><span class="p">,</span> <span class="s">"status_code"</span><span class="p">:</span> <span class="mi">0</span><span class="p">}),</span> <span class="mi">400</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s">"error"</span><span class="p">:</span> <span class="s">"Invalid Token"</span><span class="p">,</span> <span class="s">"status_code"</span><span class="p">:</span> <span class="mi">0</span><span class="p">}),</span> <span class="mi">400</span>
<span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span><span class="s">"error"</span><span class="p">:</span> <span class="s">"Authentication failed"</span><span class="p">,</span> <span class="s">"status_code"</span><span class="p">:</span> <span class="mi">0</span><span class="p">}),</span> <span class="mi">400</span>
<span class="k">return</span> <span class="n">wrap</span>
</pre></div>
<p>Now, let us say we have a view called get_profile, and say for instance that view can be called only after the above authentication is passed, then we can add the decorator just above the get_profile view definition as shown below.</p>
<div class="codehilite"><pre>@login_required
def update_user_profile():
request_data = request.form
user = User()
response = user.update_user_profile(request_data=request_data, files_data=request.files)
return myresponse(response)
</pre></div>
<h3>Decorator usage for authorization</h3>
<p>Just like how we created a decorator for authentication, we can create one more for authorization as shown below.</p>
<div class="codehilite"><pre><span class="kn">from</span> <span class="nn">functools</span> <span class="kn">import</span> <span class="n">wraps</span>
<span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">g</span><span class="p">,</span> <span class="n">request</span><span class="p">,</span> <span class="n">redirect</span><span class="p">,</span> <span class="n">url_for</span>
<span class="k">def</span> <span class="nf">authorization_required</span><span class="p">():</span>
<span class="k">def</span> <span class="nf">decorator</span><span class="p">(</span><span class="n">f</span><span class="p">):</span>
<span class="nd">@wraps</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">wrap</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span> <span class="n">kwargs</span><span class="p">):</span>
<span class="n">user_id</span> <span class="o">=</span> <span class="n">session</span><span class="p">[</span><span class="s">"user_id"</span><span class="p">]</span>
<span class="n">org</span> <span class="o">=</span> <span class="n">validate_user</span><span class="p">(</span><span class="n">user_id</span><span class="p">)</span> <span class="o">//</span><span class="n">this</span> <span class="n">function</span> <span class="n">contains</span> <span class="n">business</span> <span class="n">specific</span> <span class="n">logic</span> <span class="n">to</span> <span class="n">validate</span> <span class="n">user</span>
<span class="k">if</span> <span class="s">"is_active"</span> <span class="ow">in</span> <span class="n">org</span><span class="p">[</span><span class="s">"data"</span><span class="p">]</span> <span class="ow">and</span> <span class="n">org</span><span class="p">[</span><span class="s">"data"</span><span class="p">][</span><span class="s">"is_active"</span><span class="p">]</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
<span class="k">return</span> <span class="n">f</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">jsonify</span><span class="p">({</span>
<span class="s">"message"</span><span class="p">:</span> <span class="n">org</span><span class="p">[</span><span class="s">"message"</span><span class="p">],</span>
<span class="s">"error"</span><span class="p">:</span> <span class="n">org</span><span class="p">[</span><span class="s">"message"</span><span class="p">],</span>
<span class="s">"status_code"</span><span class="p">:</span> <span class="mi">0</span><span class="p">}),</span> <span class="mi">400</span>
<span class="k">return</span> <span class="n">wrap</span>
<span class="k">return</span> <span class="n">decorator</span>
</pre></div>
<p>and now if the request to the view get_profile needs needs to get both authenticated and authorized, then we can add both the decorators just about the view get_profile as shown below.</p>
<div class="codehilite"><pre>@login_required
@organization_required(admin_required=False)
def get_profile(id)
request_data = request.form
user = User()
response = user.update_user_profile(request_data=request_data, files_data=request.files)
return myresponse(response)
</pre></div>Differences between python and java2020-08-03T07:05:20+00:00Balaji P/blog/author/bljd369/http://django.cowhite.com/blog/differences-between-python-and-java/<p>This article is not to discuss the strenghts and weaknesses of both python and java, rather it is to explain and then realize how powerful these two languages are.</p>
<h3>Java:</h3>
<p>If you are a java progarammer, first you need to compile the code that you have written , and this compiled code is nothing but the byte code. This byte code will be executed by the JVM at run time.</p>
<h3>Python:</h3>
<p>It is a scripting language. Each and every line will be interpreted at run time, and even the datatype determination happens at runtime, hence it is a bit slower when compared to java.</p>
<h3>Performance:</h3>
<p>If we compare the peformance between java and python, java will be ahead because type checking in case of java is static, that means the type checking of the variable will happen at compile time. </p>
<p>If there are any symantic errors in your java code, the application server cannot be started,because the type checking happens at compile time itself and this is good actually since the errors are found at initial stage itself.</p>
<p>In case of python, type checking of variables is dynamic in nature, that means the type checking of variables happens during interpretation at run time, so the application server can be started even if you have symantic errors, so that there are chances that the issues will be found only at run time in case of python. Also the processing time will be marginally higher in case of python when compared to java. </p>
<h3>Why I have chosen python over java:</h3>
<p>The above point is not at all a rude shock to the python aspirants, because web page loading is not always about computation, it is something more than that.
The webpages are all about rich user interfaces and in most case, the user interface development is the driving force. Based on the experience I have, at any time, when I think about web development , I always choose python over java and the reason for me is simple, it takes less development time when compared to java.</p>
<p>I know some people say that in case of huge enterprise applications(where the the concept of concurrency come into picture), java is better , and may be they are correct, but if the application that you are developing is not enterprise related, then you can go for python. To make web application development easier with python, we can use frameworks like django, flask.</p>Models and Migrations2017-08-03T09:53:50+00:00srinath/blog/author/srinath/http://django.cowhite.com/blog/models-migrations-and-django-admin/<h3>Setting up Database</h3>
<p>
Before we start with models let's understand how to setup database to work with Django. Django comes pre bundled and ready to work with SQLite database, which for development and for very small lightweight websites would be enough to wor with, but if the scale of the website or the user-base of the website is even moderately high, SQLite is definitely not enough.
</p>
<p><br />
<p>
Other than SQLite Django can work with the following Databases:<br />
• MySQL<br />
• Postgres<br />
• Oracle<br />
</p><br />
<p>
Let's see how to setup the database for each of the above backends.
</p><br /><br />
<h4>SQLite</h4>
<p>
The settings for any database is registered in the <strong>settings.py</strong> file in <em>myprojectname/myprojectname/</em> directory. We register them by listing the settings for the database as a Python dictionary. It is a nested dictionary whose contents map a database alias to a dictionary containing the options for an individual database.
</p><br />
<p>
For the SQLite backend we have the following setting, which is probably the simplest:</p>
<div class="codehilite"><pre>DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'awesomedb',
}
}
</pre></div>
<p></p><br />
<p>
Please note that the dictionary is assigned to a variable named <code>DATABASES</code>. It should be named the same. Also the <code>'NAME'</code> property specifies the SQLite database filename that should be used to store the backend data.
</p><br /><br />
<h4>MySQL, Postgres, Oracle</h4>
<p>
When connecting to other database backends, such as MySQL, Oracle, or PostgreSQL, additional connection parameters will be required, viz, database username and password, database host URL and port etc.
</p><br />
<p>
Also unlike SQLite database, we need to install certain dependencies for Django to be able to connect to these backends.<br />
• If you’re using PostgreSQL, you’ll need the <a href="http://initd.org/psycopg/">psycopg2</a> package (2.5.4 or higher).<br />
• If you’re using MySQL, you’ll need a DB API driver like <a href="https://pypi.python.org/pypi/mysqlclient">mysqlclient</a>. Also MySQL version should be 5.5 or higher.
• If you’re using Oracle (v11.2 or higher supported), you’ll need a copy of <a href="https://oracle.github.io/python-cx_Oracle/">cx_Oracle</a> (version 5.2 or higher).</p>
<p></p><br />
<p>
Given below is the example for connecting a <strong>PostGres</strong>:
</p><br />
<p></p>
<div class="codehilite"><pre>DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'awesomedb',
'USER': 'awesomedbuser',
'PASSWORD': 'mypassword',
'HOST': '127.0.0.1',
'PORT': '5432',
}
}
</pre></div>
<p></p><br />
<p>
For connecting to other Databases, edit the value for <code>'ENGINE'</code> as follows:<br />
</p><br />
<table>
<tbody>
<tr>
<th style="padding-left: 5px; padding-right: 5px;border-bottom: 1px solid;">
Database
</th>
<th style="padding-left: 5px; padding-right: 5px;border-bottom: 1px solid;">
<code>'ENGINE'</code> value
</th>
</tr>
<tr>
<td style="padding-left: 5px; padding-right: 5px;">
MySQL
</td>
<td style="padding-left: 5px; padding-right: 5px;">
<code>'django.db.backends.mysql'</code>
</td>
</tr>
<tr>
<td style="padding-left: 5px; padding-right: 5px;">
ORACLE
</td>
<td style="padding-left: 5px; padding-right: 5px;">
<code>'django.db.backends.oracle'</code>
</td>
</tr>
</tbody>
</table><br /><br />
<h4>Timezone Setting</h4>
<p>
While you are setting up the <strong>myprojectname/settings.py</strong>, set <code>TIME_ZONE</code> to your timezone. <code>TIME_ZONE</code> should be a string representing the time zone. Click here for the list of <a href="https://en.wikipedia.org/wiki/List_of_tz_database_time_zones">timezones</a>.
</p><br /><br />
<h4>Installed apps</h4>
<p>
Note the <code>'INSTALLED_APPS'</code> setting at the top of the file. This holds the names of all the Django applications activated in this particular Django project. As we have discussed above, apps can be used in multiple projects and can also be packaged and distributed for use by others in their projects.
</p><br />
<p>
The default <code>'INSTALLED_APPS'</code> contain the following apps. These apps are prepackaged apps that come with Django.<br />
• <code>django.contrib.admin</code> -> The admin site. We will look in to it ahead in this blog.<br />
• <code>django.contrib.auth</code> -> The authentication system for Django.<br />
• <code>django.contrib.contenttypes</code> -> A framework for content types.<br />
• <code>django.contrib.sessions</code> -> The sessions framework.<br />
• <code>django.contrib.staticfiles</code> -> Framework for managing static assets files.
</p><br />
<p>
<strong>Important Note</strong>: Some of the above applications make use of database. And right now although you may have setup the SQL server (no setting up needed for SQLite.), the database tables needed for these applications aren't migrated into them. So lets migrate them first. To do that, run the following command:
</p><br />
<p>
<pre>
<code>
python manage.py migrate
</code>
</pre>
</p><br /><br />
<h3>Creating a Model</h3>
<p>
Models are nothing but database layouts with additional metadata. Let's see how to create our first Models in Django.
</p><br />
<p>
In our simple poll app, we’ll create two models: Question and Choice. A Question has a question and a publication date. A Choice has two fields: the text of the choice and a vote tally. Each Choice is associated with a Question.
</p><br />
<p>
These concepts are represented by simple Python classes. Edit the polls/models.py file so it looks like this:
</p><br />
<p></p>
<div class="codehilite"><pre><span class="c"># Python Code</span>
<span class="c"># polls/models.py</span>
<span class="kn">from</span> <span class="nn">django.db</span> <span class="kn">import</span> <span class="n">models</span>
<span class="k">class</span> <span class="nc">Question</span><span class="p">(</span><span class="n">models</span><span class="o">.</span><span class="n">Model</span><span class="p">):</span>
<span class="n">question_text</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">CharField</span><span class="p">(</span><span class="n">max_length</span><span class="o">=</span><span class="mi">200</span><span class="p">)</span>
<span class="n">pub_date</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">DateTimeField</span><span class="p">(</span><span class="s">'date published'</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">Choice</span><span class="p">(</span><span class="n">models</span><span class="o">.</span><span class="n">Model</span><span class="p">):</span>
<span class="n">question</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">ForeignKey</span><span class="p">(</span><span class="n">Question</span><span class="p">,</span> <span class="n">on_delete</span><span class="o">=</span><span class="n">models</span><span class="o">.</span><span class="n">CASCADE</span><span class="p">)</span>
<span class="n">choice_text</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">CharField</span><span class="p">(</span><span class="n">max_length</span><span class="o">=</span><span class="mi">200</span><span class="p">)</span>
<span class="n">votes</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">IntegerField</span><span class="p">(</span><span class="n">default</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
</pre></div>
<p></p><br />
<p>
Let's dissect this block of code and understand the significance of each line.
</p><br >
<p>
Both the models are represented by a class that has <code>django.db.models.Model</code> as its parent class. And the fields (or columns, in SQL terms) are represented by class variables.
</p><br />
<p>
For the fields that are supposed to be persisted are represented by an instance of a subclass of the <code>Field</code> class - e.g., <code>CharField</code> for character fields, <code>DataTimeField</code> fot datetimes and <code>IntegerField</code> for integer fields and so on.
</p><br />
<p>
The name of each <code>Field</code> instance (e.g. <code>question_text</code> or <code>pub_date</code>) is the field’s name, in machine-friendly format. You’ll use this value in your Python code, and your database will use it as the column name.
</p><br />
<p>
Some <code>Field</code> classes have required arguments. <code>CharField</code>, for example, requires that you give it a <code>max_length</code>. That’s used not only in the database schema, but in validation, as we’ll soon see. They also have optional arguments; in our case we have set the <code>default</code> value of <code>votes</code> to 0.
</p><br />
<p>
Finally, note a relationship is defined, using <code>ForeignKey</code>. That tells Django each <code>Choice</code> is related to a single <code>Question</code>. Django supports all the common database relationships: many-to-one, many-to-many, and one-to-one.
</p><br />
<h3>Migrating the Models</h3>
<p>
Django apps are basically "pluggable" modules. So here we have given Django a lot of information with our models.py code. And based on this information Django <strong>Creates the database schema</strong> and a <strong>Python wrapper API</strong> around this database for accessing the data from the database.
</p><br />
<p>
But for all this to happen we need to tell our Django project that we want the <code>polls</code> app to be installed. And to do that we add a reference to its configuration class in the <code>'INSTALLED_APPS'</code> setting we saw earlier. The <code>PollsConfig</code> class is in the <em>polls/apps.py</em> file, so its dotted path is <code>'polls.apps.PollsConfig'</code>. Edit the <em>mysite/settings.py</em> file and add that dotted path to the <code>INSTALLED_APPS</code> setting. It’ll look like this:
</p><br />
<p></p>
<div class="codehilite"><pre># Python Code
# myprojectname/settings.py
INSTALLED_APPS = [
'polls.apps.PollsConfig',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
]
</pre></div>
<p></p><br />
<p>
This tells Django to include the <code>polls</code> app. Now let's make the migration files:
</p><br />
<p>
<pre>
<code>
python manage.py makemigrations polls
</code>
</pre>
</p><br />
<p>
You should see something similar to the following:
</p><br />
<p></p>
<div class="codehilite"><pre>Migrations for 'polls':
polls/migrations/0001_initial.py:
- Create model Choice
- Create model Question
- Add field question to choice
</pre></div>
<p></p><br />
<p>
What this does is notify Django that certain changes have been made in the models. These changes are stored in a <em>migration</em> file. This migration file is used to change the database schema. These migrations are just regular files with instructions to change the schema. We could read the migration if needed. It will be a file similar to <em>polls/migrations/0001_initial.py</em>. Reading them is not a compulsion, rather it is just to quench your thirst of curiosity.
</p><br />
<p>
These migration files do not include actual SQL statements rather it includes a intermediate python code which is used to generate the SQL commands. To view the actual SQL statements we could run the following command:
</p><br />
<p>
<pre>
<code>
python manage.py sqlmigrate polls 0001
</code>
</pre>
</p><br />
<p>
Note that the <code>0001</code> denotes the prefix of <strong>0001_initial.py</strong>
</p><br />
<p>
Let's actually write the changes to database now. This can be done by running the following command:
</p><br />
<p>
<pre>
<code>
python manage.py migrate
</code>
</pre>
</p><br />
<p>
This actually applies the changes into the database by running the migration. Django has a note of all applied migrations and runs them against the database only if not yet applied. Thus essentially syncing the changes to your database based on you your changes in Models.
</p><br />
<p>
The reason that there are separate commands to make and apply migrations is because you’ll commit migrations to your version control system and ship them with your app; they not only make your development easier, they’re also usable by other developers and in production.
</p><br />
<p>
So, the steps to remember for changing the database schema is as follows:<br />
• Change your models in <em>models.py</em> file.<br />
• Run <code>python manage.py makemigrations</code> to create migrations for the changes made.<br />
• Run <code>python manage.py migrate</code> to apply those changes to the database.
</p><br /></p>A complete guide/tutorial to Django2017-07-31T04:00:00+00:00srinath/blog/author/srinath/http://django.cowhite.com/blog/a-complete-guide-tutorial-to-django/<h3>What is Django?</h3>
<p>Django is a Python framework that make developing database driven website that can be set up in mere hours for most simple websites. The elegance of Django is that the actual coding we have to do is very less and many a boilerplate coding that every developer codes in every app is already implemented. However everything is still kept customizable and tweak-able. There is also the obvious advantage of being able to use the enormous list of Python library modules.</p>
<p><br />
<p>Django takes care of much of the hassle of web development, so you can focus on writing your app without needing to reinvent the wheel. And most importantly it's completely free and open source, which means Django has a huge following with thriving and active community.</p><br /><br />
<h3>Why choose Django?</h3>
<p>As we discussed earlier, we don't have to reinvent the wheel over and over again, nor do we have to copy paste a lot of boilerplate code that many common application development cycle needs. It already has built in modules ready to be used with just some few lines of code (in some case, as low as 1 line) and you are good to go! </p><br />
<p>Django helps you write software that is:</p><br />
<p><strong>Flexible</strong>:<br />Django can be used to build almost any type of website. Be it a Content Management System or an E commerce website, or be it just a blog, or a complex Vehicle tracking web app, Django can handle it all. Also Django can deliver this content in multiple formats, viz, HTML, JSON, XML, RSS Feeds to name some.</p><br />
<p><strong>Independent</strong>:<br />Although you could use the any python library from the plethora of python packages online, Django in itself is complete. In other words, Django includes everything it needs in one "product". It all works seamlessly together, follows consistent design principles, and has extensive and up-to-date documentation.</p><br />
<p><strong>Secure</strong>:<br />Django helps developers avoid many common security mistakes by providing a framework that has been engineered to "do the right things" to protect the website automatically. For example, Django provides a secure way to manage user accounts and passwords, avoiding common mistakes like putting session information in cookies where it is vulnerable (instead cookies just contain a key, and the actual data is stored in the database) or directly storing passwords rather than a password hash.
</p><br />
<p>
Django enables protection against many vulnerabilities by default, including SQL injection, cross-site scripting, cross-site request forgery and click-jacking (see Website security for more details of such attacks).</p><br />
<p><strong>Scalable</strong>:<br />Django due to its lack of dependencies on other packages and "shared nothing" architecture (each part of the architecture is independent of the others, and can hence be replaced or changed if needed), can be scaled for increased traffic by just adding hardware at any level: caching servers, database servers, or application servers. This is so due to its clear separation between each part or modules.</p><br />
<p><strong>Coder Friendly</strong>:<br />Django endorses the Don't Repeat Yourself (DRY) philosophy. Django itself is written that exhibit this principle. It's design principles and patterns encourage the creation of maintainable and reusable code. Thus reducing the overall load on coder and making the maintenance cycle as easy as ABC.</p><br />
<p>Django is written in Python, which runs on many platforms. That means that you are not tied to any particular server platform, and can run your applications on many flavors of Linux, Windows, and Mac OS X. Furthermore, Django is well-supported by many web hosting providers, who often provide specific infrastructure and documentation for hosting Django sites.</p><br />
<p><strong>Free & open source</strong>:<br />Django is completely free to use. It's open source too, due to which it has a huge community and active fan following. It also means that you'll never run out of pre-built library which you could append to your app and avoid having to code for every petty little thing and use what's already implemented and get on with stuff that matters.</p><br /><br />
<h3>What next?</h3>
<p>In the blogs to come, we will set up and learn Django in-depth; the only prerequisite being that you be familiar with Python syntax. We will understand the structure, coding style and best practices on the way and crush Django to our whim!</p><br />
<p>As and when any topic is added it will be added to the Index below, from which you could navigate through the whole series of tutorials.</p><br /><br />
<h2>Index</h2>
<ul>
<li>
1. <a href="https://django.cowhite.com/blog/installing-and-setting-up-django/">Installing and setting up Django</a>
</li>
<li>
2. <a href="https://django.cowhite.com/blog/writing-our-first-app-and-understanding-the-file-structure-of-django/">Writing our first app and understanding the file structure of Django</a>
</li>
<li>
3. <a href="https://django.cowhite.com/blog/models-migrations-and-django-admin/">Models and Migrations</a>
</li>
</ul></p>Python Lists, Shallow and Deep Copy2017-07-21T15:15:26+00:00ganesh/blog/author/ganesh/http://django.cowhite.com/blog/python-lists-shallow-and-deep-copy/<p>In python, as we know copying lists is not same as copying variables of data types like integer. Let us see different situations we come across while copying the lists.</p>
<p>Also the way simple lists act while copying is different from that of nested lists. Let's see what actually happens when we copy an integer variable in python.</p>
<div class="codehilite"><pre>>>> x = 4
>>>y = x
>>>print (x,y)
4 4
>>>
</pre></div>
<p>What actually happens here is, python creates a variable y which references to value 4. To understand more we make use of python id() function.</p>
<p>The id() function returns identity of the object. This is an integer which is unique for the given object and remains constant during its lifetime</p>
<div class="codehilite"><pre>>>> x = 4
>>>y = x
>>>print (x,y)
4 4
>>>print (id(x), id(y))
30503184 30503184
>>>
</pre></div>
<p>So what is happening here is, python is not creating a new integer object for y. It is creating a variable y which refers to data 4.</p>
<p>Now let's see what happens if we take the same example with list instead of integer.</p>
<div class="codehilite"><pre>>>> list1 = ['a','b','c','d']
>>> list2 = list1
>>> print list1, list2
['a', 'b', 'c', 'd'] ['a', 'b', 'c', 'd']
>>> print id(list1), id(list2)
140118925400256 140118925400256
>>> list2 = ['e','f','g','h']
>>> print list1
['a', 'b', 'c', 'd']
>>> print list2
['e', 'f', 'g', 'h']
>>>
</pre></div>
<p>This is what is called shallow copy. Python never creates real copies by default. It does shallow copy. We need to explicitly write code to make real copy.</p>
<p>The difference between real copy and shallow copy can be represented as shown in the image below.</p>
<ul>
<li></li>
</ul>
<p><img alt="alt text" src="https://django.cowhite.com/static/media/uploads/shallow%20and%20deep%20copy/python-shallow-and-deep.png" /></p>
<ul>
<li></li>
</ul>
<p>Assigning new list to "list2" has no effect on "list1". Because here, python is assigning a new list object to "list2". The problem starts when we want to change just a single element of list instead of the whole list. Continuing with above example, </p>
<div class="codehilite"><pre>>>> list1 = ['a','b','c','d']
>>> list2 = list1
>>> print list1, list2
['a', 'b', 'c', 'd'] ['a', 'b', 'c', 'd']
>>> print id(list1), id(list2)
140118925400256 140118925400256
>>> list2[1] = 'z'
>>> print list2
['a', 'z', 'c', 'd']
>>> print list1
['a', 'z', 'c', 'd']
>>>
</pre></div>
<p>As you can see, both the lists "list1" and "list2" are changed even though we just updated the value of list "list2". This is because both "list1" and "list2" are referring to the same list object. The explanation is that we didn't assign a new object to list "list2". Both variables "list1" and "list2" still point to the same list object.</p>
<p><strong>Slice operator</strong>:</p>
<p>We can avoid the above problems by using slice operator to copy lists.</p>
<div class="codehilite"><pre>>>> list1 = ['a','b','c','d']
>>> list2 = list1[:]
>>> print list1, list2
['a', 'b', 'c', 'd'] ['a', 'b', 'c', 'd']
>>> list2[1] = 'z'
>>> print list1, list2
['a', 'b', 'c', 'd'] ['a', 'z', 'c', 'd']
>>>
</pre></div>
<p>This works fine with the simple lists. What happens if we perform the similar operations on nested lists. </p>
<div class="codehilite"><pre>>>> list1 = ['a','b',['x', 'y']]
>>> list2 = list1[:]
>>> print list1
['a', 'b', ['x', 'y']]
>>> print list2
['a', 'b', ['x', 'y']]
>>> list2[0] = 'cow'
>>> print list2
['cow', 'b', ['x', 'y']]
>>> print list1
['a', 'b', ['x', 'y']]
>>>
</pre></div>
<p>If we change the value of 0th of 1st index element of the "list2", there wont be any effect on the "list1". But if we change the values within the sublist, it effects the "list1".</p>
<div class="codehilite"><pre>>>> list2[2][1] = 'wow'
>>> print list1
['a', 'b', ['x', 'wow']]
>>> print list2
['cow', 'b', ['x', 'wow']]
>>>
</pre></div>
<p><strong>DeepCopy</strong></p>
<p>To avoid all these problems, python provides a method called <em>deepcopy</em>. We will perform the above operations using <em>deepcopy</em> and see what happens.</p>
<div class="codehilite"><pre><span class="o">>>></span> <span class="kn">from</span> <span class="nn">copy</span> <span class="kn">import</span> <span class="n">deepcopy</span>
<span class="o">>>></span> <span class="n">list1</span> <span class="o">=</span> <span class="p">[</span><span class="s">'a'</span><span class="p">,</span><span class="s">'b'</span><span class="p">,[</span><span class="s">'x'</span><span class="p">,</span> <span class="s">'y'</span><span class="p">]]</span>
<span class="o">>>></span> <span class="n">list2</span> <span class="o">=</span> <span class="n">deepcopy</span><span class="p">(</span><span class="n">list1</span><span class="p">)</span>
<span class="o">>>></span> <span class="k">print</span> <span class="n">list1</span>
<span class="p">[</span><span class="s">'a'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">,</span> <span class="p">[</span><span class="s">'x'</span><span class="p">,</span> <span class="s">'y'</span><span class="p">]]</span>
<span class="o">>>></span> <span class="k">print</span> <span class="n">list2</span>
<span class="p">[</span><span class="s">'a'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">,</span> <span class="p">[</span><span class="s">'x'</span><span class="p">,</span> <span class="s">'y'</span><span class="p">]]</span>
<span class="o">>>></span> <span class="k">print</span> <span class="nb">id</span><span class="p">(</span><span class="n">list1</span><span class="p">),</span> <span class="nb">id</span><span class="p">(</span><span class="n">list2</span><span class="p">)</span>
<span class="mi">140118925529168</span> <span class="mi">140118925479880</span>
<span class="o">>>></span>
</pre></div>
<p>Using deepcopy, python creates a new list object for "list2". We can see that by noticing the identifiers of the two lists. We can also check the id's of the sublist and see that deepcopy, unlike shallow copy, creates a new object for the "list2"</p>
<div class="codehilite"><pre>>>> print id(list1[2]), id(list2[2])
140118925529240 140118925583136
>>> print id(list1[2][1]), id(list2[2][1])
140118925706504 140118925706504
>>>
</pre></div>
<p>We can also check by assigning some value to sublist of "list2" and see what happens to "list1".</p>
<div class="codehilite"><pre>>>> list2[2][1] = "wow"
>>> print list1
['a', 'b', ['x', 'y']]
>>> print list2
['a', 'b', ['x', 'wow']]
>>>
</pre></div>
<p>Deepcopy provides an elegant solution for the list copying but one should notice that it takes up extra space even in cases whn it not needed. One should carefully use the deepcopy method in ordered reduce the memory usage.</p>Python and the Context Managers2017-07-17T08:56:59+00:00ravi/blog/author/ravi/http://django.cowhite.com/blog/python-and-the-context-manager/<p><a href="https://docs.python.org/2/reference/datamodel.html#context-managers" target="_blank">Context managers</a> is one of those features of Python that makes it extremely easy to write clean code. Context managers are extensively used by Python programmers, but very few understand how they work. Let us explore them a bit.</p>
<p><br> </p>
<p>Every python developer knows that the right way of working with files in Python is.</p>
<div class="codehilite"><pre>with open('myfile.txt', 'w') as my_file:
do_something_with(my_file)
</pre></div>
<p>And of course we also know that this is equivalent to </p>
<div class="codehilite"><pre>my_file = open('myfile.txt', 'w')
try:
do_something_with(my_file)
finally:
my_file.close()
</pre></div>
<p>That is ok. But why and how does it work?</p>
<h3>What is a context manager?</h3>
<p>A context manager in Python is an object that implements <code>__enter__(self)</code> and <code>__exit__(self, type, value, traceback)</code> methods. Once our class implements those two methods, we can use it as a context manager with the <code>with</code> statement. But how does that work? When the control enters the <code>with</code> statement block, <code>with</code> statement will run the <code>__enter__</code> method of the context manager. And once the code within the <code>with</code> block is done executing, the <code>__exit__</code> method of the context manager is executed. We can demonstrate this by creating our own context manager. Let us define our class <code>MyContextManager</code> as follows.</p>
<div class="codehilite"><pre><span class="k">class</span> <span class="n">MyContextManager</span>():
<span class="n">def</span> <span class="n">__enter__</span>(<span class="k">self</span>):
<span class="nb">print</span>(<span class="s">'The context has been setup.'</span>)
<span class="n">def</span> <span class="n">__exit__</span>(<span class="k">self</span>, <span class="n">type</span>, <span class="nb">value</span>, <span class="n">traceback</span>):
<span class="nb">print</span>(<span class="s">'Exiting the context.'</span>)
</pre></div>
<p>We use this class in a <code>with</code> statement as follows.</p>
<div class="codehilite"><pre>with MyContextManager():
print('Within the context.')
</pre></div>
<p>The output of the statement is</p>
<div class="codehilite"><pre>The context has been setup.
Within the context.
Exiting the context.
</pre></div>
<p>This is how <code>with</code> statement makes sure that the resources are released once the block is complete. Let us see how we can make a context manager to work with files (ignoring the fact that the file descriptor already is a context manager).</p>
<p><br></p>
<p>Points to note before creating a context manager.
<br></p>
<p><strong>1.</strong> The <code>__enter__</code> method of the context manager does not take any arguments. The value returned by the <code>__enter__</code> method can be used with <code>as</code> statement within <code>with</code> statement (as in <code>with open('myfile.txt', 'w') as f:</code>).
<br>
<strong>2.</strong> The <code>__exit__</code> method will take 3 arguments, the class type of the exception, the value of the exception and the traceback. All these arguments will be <code>None</code> if there is no exception raised in the <code>with</code> block.
<br>
<strong>3.</strong> If the <code>__exit__</code> method return <code>True</code>, the <code>with</code> statement will suppress the error (does not propagate the error). Otherwise, the exception is thrown by the <code>with</code> block.</p>
<p>Now our context manager for file descriptor will be</p>
<div class="codehilite"><pre><span class="k">class</span> <span class="n">File</span>():
<span class="n">def</span> <span class="n">__init__</span>(<span class="k">self</span>, <span class="nb">name</span>, <span class="n">mode</span>):
<span class="k">self</span>.<span class="n">file</span> = <span class="nb">open</span>(<span class="nb">name</span>, <span class="n">mode</span>)
<span class="n">def</span> <span class="n">__enter__</span>(<span class="k">self</span>):
<span class="nb">print</span>(<span class="s">'The file will be returned.'</span>)
<span class="k">return</span> <span class="k">self</span>.<span class="n">file</span>
<span class="n">def</span> <span class="n">__exit__</span>(<span class="k">self</span>, <span class="n">type</span>, <span class="nb">value</span>, <span class="n">traceback</span>):
<span class="nb">print</span>(<span class="s">'The file will be closed now.'</span>)
<span class="k">self</span>.<span class="n">file</span>.<span class="nb">close</span>()
<span class="k">return</span> <span class="nb">True</span>
</pre></div>
<p>This can be used with <code>with</code> statement like this.</p>
<div class="codehilite"><pre>with File('testfile.txt', 'w') as myfile:
myfile.write('Hello world!')
</pre></div>
<p>The output on the console will is </p>
<div class="codehilite"><pre>The file will be returned.
The file will be closed now.
</pre></div>
<p>The result of the above statements is that a new file named <code>testfile.txt</code> will be opened (it will be created first if it does not exist already), the line <code>Hello world!</code> will be written to it. Once the <code>with</code> block is complete, the file will be closed.</p>
<p><br></p>
<p>Let us also examine how exceptions are handled by the context managers and <code>with</code> statement. Lets run the following statements.</p>
<div class="codehilite"><pre>with File('testfile.txt', 'w') as myfile:
myfile.write('Hello world!')
raise Exception('Could this exception create a problem?')
print('Exception is handled well')
</pre></div>
<p>Here we are raising an exception from within the <code>with</code> block (which is a possibility that we actually want to solve with context managers). The output is </p>
<div class="codehilite"><pre>The file will be returned.
The file will be closed now.
Exception is handled well
</pre></div>
<p>Since the <code>__exit__</code> method returns <code>True</code>, the exception is not propagated by <code>with</code>. Hence the final <code>print</code> statement is executed. If the context manager is changed to </p>
<div class="codehilite"><pre><span class="k">class</span> <span class="n">File</span>():
<span class="n">def</span> <span class="n">__init__</span>(<span class="k">self</span>, <span class="nb">name</span>, <span class="n">mode</span>):
<span class="k">self</span>.<span class="n">file</span> = <span class="nb">open</span>(<span class="nb">name</span>, <span class="n">mode</span>)
<span class="n">def</span> <span class="n">__enter__</span>(<span class="k">self</span>):
<span class="nb">print</span>(<span class="s">'The file will be returned.'</span>)
<span class="k">return</span> <span class="k">self</span>.<span class="n">file</span>
<span class="n">def</span> <span class="n">__exit__</span>(<span class="k">self</span>, <span class="n">type</span>, <span class="nb">value</span>, <span class="n">traceback</span>):
<span class="nb">print</span>(<span class="s">'The file will be closed now.'</span>)
<span class="k">self</span>.<span class="n">file</span>.<span class="nb">close</span>()
<span class="k">return</span> <span class="nb">False</span>
</pre></div>
<p>Notice that the <code>__exit__</code> method returns <code>False</code> which instructs the <code>with</code> statement not to handle the exception (but propagate the exception). The result is</p>
<div class="codehilite"><pre><span class="nx">The</span> <span class="nb">file</span> <span class="nx">will</span> <span class="nx">be</span> <span class="nx">returned.</span>
<span class="nx">The</span> <span class="nb">file</span> <span class="nx">will</span> <span class="nx">be</span> <span class="nx">closed</span> <span class="nx">now.</span>
<span class="o">---------------------------------------------------------------------------</span>
<span class="nx">Exception</span> <span class="nx">Traceback</span> <span class="p">(</span><span class="nx">most</span> <span class="nx">recent</span> <span class="nx">call</span> <span class="nx">last</span><span class="p">)</span>
<span class="o"><</span><span class="nx">ipython</span><span class="na">-input</span><span class="o">-</span><span class="mi">14</span><span class="na">-f7f6fb4fb916</span><span class="o">></span> <span class="k">in</span> <span class="o"><</span><span class="nx">module</span><span class="o">></span><span class="p">()</span>
<span class="mi">1</span> <span class="k">with</span> <span class="nb">File</span><span class="p">(</span><span class="s1">'testfile.txt'</span><span class="p">,</span> <span class="s1">'w'</span><span class="p">)</span> <span class="nx">as</span> <span class="nx">myfile</span><span class="p">:</span>
<span class="mi">2</span> <span class="nx">myfile.write</span><span class="p">(</span><span class="s1">'Hello world!'</span><span class="p">)</span>
<span class="o">----></span> <span class="mi">3</span> <span class="nx">raise</span> <span class="nx">Exception</span><span class="p">(</span><span class="s1">'Could this exception create a problem?'</span><span class="p">)</span>
<span class="mi">4</span> <span class="nx">print</span><span class="p">(</span><span class="s1">'Exception is handled well'</span><span class="p">)</span>
<span class="nx">Exception</span><span class="p">:</span> <span class="nx">Could</span> <span class="nx">this</span> <span class="nx">exception</span> <span class="nx">create</span> <span class="nx">a</span> <span class="nx">problem</span><span class="o">?</span>
</pre></div>
<p>When the <code>__exit__</code> returns <code>False</code>, the exception is propagated to the Python interpreter.</p>
<h3>Creating context managers with <a href="https://docs.python.org/2/library/contextlib.html" target="_blank"><code>contextlib</code></a></h3>
<p>Python has a complete module called <a href="https://docs.python.org/2/library/contextlib.html" target="_blank"><code>contextlib</code></a> dedicated to create and manage context managers. We have already seen a way of creating context manager with a class implementing the <code>__enter__</code> and <code>__exit__</code> methods. Using the <code>contextlib.contextmanager</code> is another way to create a context manager.</p>
<p><br></p>
<p>The <code>contextlib.contextmanager</code> decorator can be used on an iterator function that yields exactly on value. It means the function must use exactly one yield statement. Every statement before the <code>yield</code> statement in the function is considered as part of the <code>__enter__</code> method and everything after that is considered as part of the <code>__exit__</code> method. The value that should be returned by the <code>__enter__</code> method must be yielded. This is how we implement our <code>File</code> (our example class) context manager with the <code>contextlib.contextmanager</code> decorator.</p>
<div class="codehilite"><pre><span class="kn">from</span> <span class="nn">contextlib</span> <span class="kn">import</span> <span class="n">contextmanager</span>
<span class="nd">@contextmanager</span>
<span class="k">def</span> <span class="nf">file</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">mode</span><span class="p">):</span>
<span class="n">my_file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">mode</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="s">'The file is opened.'</span><span class="p">)</span>
<span class="k">yield</span> <span class="n">my_file</span>
<span class="k">print</span><span class="p">(</span><span class="s">'The file will be closed now.'</span><span class="p">)</span>
<span class="n">my_file</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>
<p>Notice that we cannot return a value in a generator. So this is always equivalent to returning <code>False</code> in the <code>__exit__</code> method. Now with our context manager <code>file</code> defined, let us run the following statements.</p>
<div class="codehilite"><pre>with file('testfile.txt', 'w') as myfile:
myfile.write('Hellooooo world!')
</pre></div>
<p>The output is </p>
<div class="codehilite"><pre>The file is opened.
The file will be closed now.
</pre></div>
<p>The takeaway here is that the files opened are closed successfully. But how does it handle the exceptions raised in the <code>with</code> block. Running the statments</p>
<div class="codehilite"><pre>with file('testfile.txt', 'w') as myfile:
myfile.write('Hellooooo world!')
raise Exception('Could this cause a problem?')
print('Exception is handled well')
</pre></div>
<p>results in the following exception.</p>
<div class="codehilite"><pre><span class="nx">The</span> <span class="nb">file</span> <span class="nx">is</span> <span class="nx">opened.</span>
<span class="o">---------------------------------------------------------------------------</span>
<span class="nx">Exception</span> <span class="nx">Traceback</span> <span class="p">(</span><span class="nx">most</span> <span class="nx">recent</span> <span class="nx">call</span> <span class="nx">last</span><span class="p">)</span>
<span class="o"><</span><span class="nx">ipython</span><span class="na">-input</span><span class="o">-</span><span class="mi">50</span><span class="o">-</span><span class="mi">58</span><span class="nx">f247e24470</span><span class="o">></span> <span class="k">in</span> <span class="o"><</span><span class="nx">module</span><span class="o">></span><span class="p">()</span>
<span class="mi">1</span> <span class="k">with</span> <span class="nb">file</span><span class="p">(</span><span class="s1">'testfile.txt'</span><span class="p">,</span> <span class="s1">'w'</span><span class="p">)</span> <span class="nx">as</span> <span class="nx">myfile</span><span class="p">:</span>
<span class="mi">2</span> <span class="nx">myfile.write</span><span class="p">(</span><span class="s1">'Hellooooo world!'</span><span class="p">)</span>
<span class="o">----></span> <span class="mi">3</span> <span class="nx">raise</span> <span class="nx">Exception</span><span class="p">(</span><span class="s1">'Could this cause a problem?'</span><span class="p">)</span>
<span class="mi">4</span> <span class="nx">print</span><span class="p">(</span><span class="s1">'Exception is handled well'</span><span class="p">)</span>
<span class="nx">Exception</span><span class="p">:</span> <span class="nx">Could</span> <span class="nx">this</span> <span class="nx">cause</span> <span class="nx">a</span> <span class="nx">problem</span><span class="o">?</span>
</pre></div>
<p>Observe that the print statement post the <code>yield</code> statement is not executed. This exception will fail our context manager to close the file descriptors. A more sophisticated way to handle this is to rewrite the context manager as</p>
<div class="codehilite"><pre><span class="kn">from</span> <span class="nn">contextlib</span> <span class="kn">import</span> <span class="n">contextmanager</span>
<span class="nd">@contextmanager</span>
<span class="k">def</span> <span class="nf">file</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">mode</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">my_file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">mode</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="s">'The file is opened.'</span><span class="p">)</span>
<span class="k">yield</span> <span class="n">my_file</span>
<span class="k">finally</span><span class="p">:</span>
<span class="k">print</span><span class="p">(</span><span class="s">'The file will be closed now.'</span><span class="p">)</span>
<span class="n">my_file</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>
<p>Now the result of the previous statements is</p>
<div class="codehilite"><pre><span class="nx">The</span> <span class="nb">file</span> <span class="nx">is</span> <span class="nx">opened.</span>
<span class="nx">The</span> <span class="nb">file</span> <span class="nx">will</span> <span class="nx">be</span> <span class="nx">closed</span> <span class="nx">now.</span>
<span class="o">---------------------------------------------------------------------------</span>
<span class="nx">Exception</span> <span class="nx">Traceback</span> <span class="p">(</span><span class="nx">most</span> <span class="nx">recent</span> <span class="nx">call</span> <span class="nx">last</span><span class="p">)</span>
<span class="o"><</span><span class="nx">ipython</span><span class="na">-input</span><span class="o">-</span><span class="mi">52</span><span class="o">-</span><span class="mi">58</span><span class="nx">f247e24470</span><span class="o">></span> <span class="k">in</span> <span class="o"><</span><span class="nx">module</span><span class="o">></span><span class="p">()</span>
<span class="mi">1</span> <span class="k">with</span> <span class="nb">file</span><span class="p">(</span><span class="s1">'testfile.txt'</span><span class="p">,</span> <span class="s1">'w'</span><span class="p">)</span> <span class="nx">as</span> <span class="nx">myfile</span><span class="p">:</span>
<span class="mi">2</span> <span class="nx">myfile.write</span><span class="p">(</span><span class="s1">'Hellooooo world!'</span><span class="p">)</span>
<span class="o">----></span> <span class="mi">3</span> <span class="nx">raise</span> <span class="nx">Exception</span><span class="p">(</span><span class="s1">'Could this cause a problem?'</span><span class="p">)</span>
<span class="mi">4</span> <span class="nx">print</span><span class="p">(</span><span class="s1">'Exception is handled well'</span><span class="p">)</span>
<span class="nx">Exception</span><span class="p">:</span> <span class="nx">Could</span> <span class="nx">this</span> <span class="nx">cause</span> <span class="nx">a</span> <span class="nx">problem</span><span class="o">?</span>
</pre></div>
<p>This will successfully close the files even if an exception is raised. I hope this post helps you understand and use the context managers better. Cheers!</p>Logging in Django2017-07-16T19:40:54+00:00srinath/blog/author/srinath/http://django.cowhite.com/blog/logging-in-django/<h3>Introduction</h3>
<p>
Logging is a very important aspect of software development and maintenance. It helps out the developer point out the bugs, errors, point of failures and potential vulnerabilities of an application. Thus it was but obvious that Django comes bundled with its own logging module.
</p>
<p><br />
<p>
Django uses Python's builtin logging module to perform system logging. Hence we can find the detailed documentation of the same in Python's own docs as well as Django's extra awesome docs. But for the sake of it, let's get to know Python's logging framework before we get into its <a href="#usage">usage in Django</a>.
</p><br /></p>
<h3>Objects of <em>logging</em></h3>
<p>
A Python logging configuration consists of four parts:
<ul>
<li><a href="#loggers">Loggers</a>
<li><a href="#handlers">Handlers</a>
<li><a href="#filters">Filters</a>
<li><a href="#formatters">Formatters</a>
</ul>
</p>
<p><br /></p>
<h4 id="loggers"><strong>Loggers</strong></h4>
<p>
A Logger is an entry point into the logging system. You can consider them as containers into which messages or logs can be pushed for processing.
</p>
<p><br />
<p>
A Logger has different <em>log levels</em>. These levels describes the gravity of the message that the logger has to handle. The Different <em>log levels</em> that can be assigned to messages are as follows:</p>
<ul>
<li><strong>DEBUG</strong>: Low level system information for the sake of debugging and assistance for the developer.
<li><strong>INFO</strong>: General system information.
<li><strong>WARNING</strong>: Information describing a minor problem that has occurred or something unfavourable or situations that are ill advised has occurred.
<li><strong>ERROR</strong>: Information describing a major problem that has occurred or something has broken the code, or logic and the execution was aborted.
<li><strong>CRITICAL</strong>: Information describing a critical problem that has occurred which may effectively put the entire application in jeopardy, or risk data corruption or loss.
</ul>
<p></p><br />
<p>
Each message that is written to the logger is a Log Record. Each log record also has a log level indicating the severity of that specific message. A log record can also contain useful metadata that describes the event that is being logged. This can include details such as a stack trace or an error code.
</p><br />
<p>
When a message is given to the logger, the log level of the message is compared to the log level of the logger. If the log level of the message meets or exceeds the log level of the logger itself, the message will undergo further processing. If it doesn’t, the message will be ignored. <br />
</p><br />
<p>
Once the Logger is done with the message, it passes the message to a <em>Handler</em>
</p><br /></p>
<h4 id="handlers"><strong>Handlers</strong></h4>
<p>
The handler is the brains, which determines the action for each message in the logger. It decides whether to write the message to the console, or file or to a network stream.
</p>
<p><br />
<p>
Just like the loggers, handlers also have a log level. The handler checks if a particular log record meets or exceeds handler's own level; if it does not then the handler gracefully ignores the message and moves on.
</p><br />
<p>
Multiple handlers can be registered for a logger, with different log level. Thus we could achieve different notification pathway depending on the gravity of the log message.
</p><br />
<p>
For eg: we could shoot a mail to the web admin for logs with levels <strong>ERROR</strong> or <strong>CRITICAL</strong>, while a second handler for appending the message to a file with log levels other than the former ones, for later inspection and analysis.
</p><br /></p>
<h4 id="filters"><strong>Filters</strong></h4>
<p>
As the name suggests, a Filter checks for certain conditions and then executes an appropriate choice of option as needed.
</p>
<p><br />
<p>
The default mode of handling is by the log level of the log message. However, by installing a filter, we can place additional filtering and criteria on logging process, thus it provides additional control over which log records are passed from logger to handler.
</p><br />
<p>
For eg: if we wish to omit <strong>ERROR</strong> messages originating from some particular source, from being sent as mail to web master.
</p><br />
<p>
Filters can also be used to modify the logging record prior to being emitted. For example, you could write a filter that downgrades ERROR log records to WARNING records if a particular set of criteria are met.
</p><br />
<p>
Filters can be installed on loggers or on handlers; multiple filters can be used in a chain to perform multiple filtering actions.
</p><br /></p>
<h4 id="formatters"><strong>Formatters</strong></h4>
<p>
Ultimately, a log record needs to be rendered as text. Formatters describe the exact format of that text. A formatter usually consists of a Python formatting string containing <a href="https://docs.python.org/3/library/logging.html#logrecord-attributes">LogRecord attributes</a>; however, you can also write custom formatters to implement specific formatting behavior.
</p>
<p><br /></p>
<h3 id="usage">Django Logging - Usage</h3>
<p>
After configuring our loggers, handlers, filters and formatters (<a href="#configure">Explained later</a>), we need to place logging calls into our code at appropriate locations. Using logging framework is simple, as follows:
</p>
<p><br />
<p></p>
<div class="codehilite"><pre><span class="c"># Python Code</span>
<span class="c"># import the logging library</span>
<span class="kn">import</span> <span class="nn">logging</span>
<span class="c"># Get an instance of a logger</span>
<span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="n">__name__</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">my_view</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">arg1</span><span class="p">,</span> <span class="n">arg</span><span class="p">):</span>
<span class="c"># Do something here...</span>
<span class="c"># yes anything!</span>
<span class="k">if</span> <span class="n">bad_apples</span><span class="p">:</span>
<span class="c"># Log an error message</span>
<span class="n">logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s">'Some bad apples in the pile!!!'</span><span class="p">)</span>
</pre></div>
<p>That's all! Every time bad_apples condition is activated, a log record with log level <strong>ERROR</strong> will be written (to a location, mentioned in the handler).
</p><br />
<p>
The <code>logging.getLogger( _ <em> name _ </em> )</code> command obtains (creating first, if necessary) an instance of a logger. The <code> _ <em> name _ </em> </code> argument is the name of the python module, which will help you location the module very easily, but if you have a better naming scheme, to identify your logger, you could use that too (as a dot-separated string).
</p><br />
<p></p>
<div class="codehilite"><pre># Python Code
# Get an instance of a specific named logger
logger = logging.getLogger('universe.galaxy.star')
</pre></div>
<p>The dotted paths of logger names define a hierarchy. The <code>universe.galaxy</code> logger is considered to be a parent of the <code>universe.galaxy.star</code> logger; the universe logger is a parent of the <code>universe.galaxy</code> logger.
</p><br />
<p>
Why is the hierarchy important? Well, because loggers can be set to propagate their logging calls to their parents. In this way, you can define a single set of handlers at the root of a logger tree, and capture all logging calls in the subtree of loggers. A logging handler defined in the <code>universe</code> namespace will catch all logging messages issued on the <code>universe.galaxy</code> and <code>universe.galaxy.star</code> loggers.
</p><br />
<p>
This propagation can be controlled on a per-logger basis. If you don’t want a particular logger to propagate to its parents, you can turn off this behaviour.
</p><br /></p>
<h4><strong>Django Logging Calls</strong></h4>
<p>
The logger instance contains an entry method for each of the default log levels:
<ul>
<li><code>logger.debug()</code>
<li><code>logger.info()</code>
<li><code>logger.warning()</code>
<li><code>logger.error()</code>
<li><code>logger.critical()</code>
</ul>
</p>
<p><br />
<p>
There are two other logging calls available:
<ul>
<li><code>logger.log()</code>: Manually emits a logging message with a specific log level.
<li><code>logger.exception()</code>: Creates an ERROR level logging message wrapping the current exception stack frame.
</ul>
</p><br /></p>
<h3 id="configure">Configuring Logging</h3>
<p>
Just putting the logging calls in your code is not enough. We need to configure the loggers, handlers, filters and formatters that we explained above too. Because these decide the action to be taken and the formatting of the message to be displayed/written to the appropriate output.
</p>
<p><br />
<p>
Python’s logging library provides several techniques to configure logging, ranging from a programmatic interface to configuration files. By default, Django uses the <a href="https://docs.python.org/3/library/logging.config.html#logging-config-dictschema">dictConfig format</a>.
</p><br />
<p>
In order to configure logging, you use LOGGING to define a dictionary of logging settings. These settings describes the loggers, handlers, filters and formatters that you want in your logging setup, and the log levels and other properties that you want those components to have.
</p><br />
<p>
By default, the LOGGING setting is merged with Django’s default logging configuration using the following scheme.
</p><br />
<p>
If the <strong>disable_existing_loggers</strong> key in the <strong>LOGGING</strong> dictConfig is set to <strong>True</strong> (which is the default) then all loggers from the default configuration will be disabled. Disabled loggers are not the same as removed; the logger will still exist, but will silently discard anything logged to it, not even propagating entries to a parent logger. Thus you should be very careful using <strong>'disable_existing_loggers'</strong>: <strong>True</strong>; it’s probably not what you want. Instead, you can set <strong>disable_existing_loggers</strong> to <strong>False</strong> and redefine some or all of the default loggers; or you can set <strong>LOGGING_CONFIG</strong> to <strong>None</strong> and handle logging config yourself.
</p><br />
<p>
Logging is configured as part of the general Django <strong>setup()</strong> function. Therefore, you can be certain that loggers are always ready for use in your project code.
</p><br /></p>
<h4><strong>Example</strong></h4>
<p>
The following example is just to give you a taste of the dictConfig format. For a detailed documentation of the same please visit the <a href="https://docs.python.org/3/library/logging.config.html#logging-config-dictschema">dictConfig format documentation</a>.
</p>
<p><br />
<p></p>
<div class="codehilite"><pre># Python Code
# myproject/myproject/settings.py
#
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'file': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': '/path/to/django/debug.log',
},
},
'loggers': {
'django': {
'handlers': ['file'],
'level': 'DEBUG',
'propagate': True,
},
},
}
</pre></div>
<p><strong>PS:</strong> Don't forget to change the <code>/path/to/django/</code> to a location where the django application has permission to write and modify files.
</p><br /></p>Dynamic fields in Django Rest Framwork serializers2017-07-05T21:04:09+00:00ravi/blog/author/ravi/http://django.cowhite.com/blog/dynamic-fields-in-django-rest-framwork-serializers/<p>Often we need to have custom fields that must be calculated dynamically. In Entity-Relationship model notation the fields are called
<strong><em>Derived Attributes</em></strong>. Such fields are derived from other fields of the same entity (model) and they might change over time (age, for example). We can create dynamic fields in django in more than one way.</p>
<p><strong> 1. Dynamic field as a model property</strong></p>
<p>Let us create a model <code>Author</code> as follows.</p>
<div class="codehilite"><pre><span class="kn">from</span> <span class="nn">django.db</span> <span class="kn">import</span> <span class="n">models</span>
<span class="kn">from</span> <span class="nn">django.utils</span> <span class="kn">import</span> <span class="n">timezone</span>
<span class="k">class</span> <span class="nc">Author</span><span class="p">(</span><span class="n">models</span><span class="o">.</span><span class="n">Model</span><span class="p">):</span>
<span class="n">name</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">CharField</span><span class="p">(</span><span class="n">max_length</span><span class="o">=</span><span class="mi">40</span><span class="p">)</span>
<span class="n">dob</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">DateField</span><span class="p">(</span><span class="n">verbose_name</span><span class="o">=</span><span class="s">'Date of birth'</span><span class="p">)</span>
</pre></div>
<p>We need a field that gives us the age. An obvious choice for this is to create a method that returns the age calculated based on the <code>dob</code> field value. But Python has the <a href="https://docs.python.org/2/library/functions.html#property">property decorator</a> that allows us to access a class methods as if it is a class attribute. It also allows us to write setter and getter for the attribute, but our case is simple and we can use it to create a read-only property that can act as a field.</p>
<p>Add the following method along with the decorator.</p>
<div class="codehilite"><pre>@property
def age(self):
return timezone.now().year - self.dob.year
</pre></div>
<p>Finally the model looks like this.</p>
<div class="codehilite"><pre><span class="kn">from</span> <span class="nn">django.db</span> <span class="kn">import</span> <span class="n">models</span>
<span class="kn">from</span> <span class="nn">django.utils</span> <span class="kn">import</span> <span class="n">timezone</span>
<span class="k">class</span> <span class="nc">Author</span><span class="p">(</span><span class="n">models</span><span class="o">.</span><span class="n">Model</span><span class="p">):</span>
<span class="n">name</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">CharField</span><span class="p">(</span><span class="n">max_length</span><span class="o">=</span><span class="mi">40</span><span class="p">)</span>
<span class="n">dob</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">DateField</span><span class="p">(</span><span class="n">verbose_name</span><span class="o">=</span><span class="s">'Date of birth'</span><span class="p">)</span>
<span class="nd">@property</span>
<span class="k">def</span> <span class="nf">age</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="n">timezone</span><span class="o">.</span><span class="n">now</span><span class="p">()</span><span class="o">.</span><span class="n">year</span> <span class="o">-</span> <span class="bp">self</span><span class="o">.</span><span class="n">dob</span><span class="o">.</span><span class="n">year</span>
</pre></div>
<p>This property (method) will be treated as if it is a field of a model by Django and Django Rest Framework serializer. Create the following serializer for the model.</p>
<div class="codehilite"><pre><span class="kn">from</span> <span class="nn">rest_framework</span> <span class="kn">import</span> <span class="n">serializers</span>
<span class="kn">from</span> <span class="nn">.models</span> <span class="kn">import</span> <span class="n">Post</span><span class="p">,</span> <span class="n">Author</span>
<span class="k">class</span> <span class="nc">AuthorSerializer</span><span class="p">(</span><span class="n">serializers</span><span class="o">.</span><span class="n">ModelSerializer</span><span class="p">):</span>
<span class="k">class</span> <span class="nc">Meta</span><span class="p">:</span>
<span class="n">model</span> <span class="o">=</span> <span class="n">Author</span>
<span class="n">fields</span> <span class="o">=</span> <span class="p">(</span><span class="s">'id'</span><span class="p">,</span> <span class="s">'name'</span><span class="p">,</span> <span class="s">'dob'</span><span class="p">,</span> <span class="s">'age'</span><span class="p">)</span>
</pre></div>
<p>Note that <code>age</code> field has to be explicitly mentioned. If <code>fields</code> is set to <code>'__all__'</code> string, the serializer does not include the <code>age</code> field. </p>
<p><img alt="Author serializer" src="http://django.cowhite.com/static/media/uploads/Serailizers%20and%20custom%20fields/screenshot_from_2017-07-05_18-53-37-min.png" /></p>
<p><strong>2. Using <code>SerializerMethodField</code></strong></p>
<p>Django Rest Framework gives us the <code>SerializerMethodField</code> field whose value is dynamically evaluated by the serializer itself and not the model. Using this method has an advantage because the value can be generated in the context of the current session.</p>
<p>Let us revert back to the original <code>Author</code> model that does not include the <code>age</code> property.</p>
<div class="codehilite"><pre><span class="k">class</span> <span class="n">Author</span>(<span class="n">models</span>.<span class="n">Model</span>):
<span class="nb">name</span> = <span class="n">models</span>.<span class="n">CharField</span>(<span class="n">max_length</span>=<span class="mi">40</span>)
<span class="n">dob</span> = <span class="n">models</span>.<span class="n">DateField</span>(<span class="n">verbose_name</span>=<span class="s">'Date of birth'</span>)
</pre></div>
<p>We use the serializer to calculate the age and pass it as a serializer field.</p>
<div class="codehilite"><pre><span class="k">class</span> <span class="n">AuthorSerializer</span>(<span class="n">serializers</span>.<span class="n">ModelSerializer</span>):
<span class="n">age</span> = <span class="n">serializers</span>.<span class="n">SerializerMethodField</span>()
<span class="k">class</span> <span class="n">Meta:</span>
<span class="n">model</span> = <span class="n">Author</span>
<span class="n">fields</span> = <span class="s">'__all__'</span>
<span class="n">def</span> <span class="n">get_age</span>(<span class="k">self</span>, <span class="n">instance</span>):
<span class="k">return</span> <span class="n">datetime</span>.<span class="n">datetime</span>.<span class="n">now</span>().<span class="n">year</span> - <span class="n">instance</span>.<span class="n">dob</span>.<span class="n">year</span>
</pre></div>
<p>Any <code>SerializerMethodField</code> will look for a <code>get_<field_name></code> method and use it as source. This is equivalent to our previous procedure.</p>
<p><img alt="Author serializer" src="http://django.cowhite.com/static/media/uploads/Serailizers%20and%20custom%20fields/screenshot_from_2017-07-05_18-53-37-min.png" /></p>
<p>We can also use the current session in the serializer. Let us change the method to show the age only to admins. Show 'hidden' for other users. Change the <code>get_age</code> method as to the following.</p>
<div class="codehilite"><pre>def calculate_age(self, instance):
request = self.context.get('request')
user = request.user
if user.is_authenticated() and user.is_admin:
return datetime.datetime.now().year - instance.dob.year
return 'Hidden'
</pre></div>
<p>and the age field to <code>age = serializers.SerializerMethodField(method_name='calculate_age')</code>. The full serializer looks like this.</p>
<div class="codehilite"><pre><span class="k">class</span> <span class="n">AuthorSerializer</span>(<span class="n">serializers</span>.<span class="n">ModelSerializer</span>):
<span class="n">age</span> = <span class="n">serializers</span>.<span class="n">SerializerMethodField</span>(<span class="n">method_name</span>=<span class="s">'calculate_age'</span>)
<span class="k">class</span> <span class="n">Meta:</span>
<span class="n">model</span> = <span class="n">Author</span>
<span class="n">fields</span> = (<span class="s">'id'</span>, <span class="s">'name'</span>, <span class="s">'dob'</span>, <span class="s">'age'</span>)
<span class="n">def</span> <span class="n">calculate_age</span>(<span class="k">self</span>, <span class="n">instance</span>):
<span class="n">request</span> = <span class="k">self</span>.<span class="nb">context</span>.<span class="n">get</span>(<span class="s">'request'</span>)
<span class="n">user</span> = <span class="n">request</span>.<span class="n">user</span>
<span class="k">if</span> <span class="n">user</span>.<span class="n">is_authenticated</span>() <span class="o">and</span> <span class="n">user</span>.<span class="n">is_staff:</span>
<span class="k">return</span> <span class="n">datetime</span>.<span class="n">datetime</span>.<span class="n">now</span>().<span class="n">year</span> - <span class="n">instance</span>.<span class="n">dob</span>.<span class="n">year</span>
<span class="k">return</span> <span class="s">'Hidden'</span>
</pre></div>
<p>The result for any user (including AnonymousUser) is as follows.</p>
<p><img alt="Serializer for AnonymousUser" src="http://django.cowhite.com/static/media/uploads/Serailizers%20and%20custom%20fields/screenshot_from_2017-07-05_19-06-53-min_(1).png" /></p>
<p>The result for admin looks like this.</p>
<p><img alt="Serializer for admin" src="http://django.cowhite.com/static/media/uploads/Serailizers%20and%20custom%20fields/screenshot_from_2017-07-05_18-53-37-min.png" /></p>
<p>Surely there are more ways to generate dynamic fields, using <a href="https://docs.djangoproject.com/en/dev/topics/db/aggregation/#generating-aggregates-for-each-item-in-a-queryset">annotations</a> for example. These are the simplest ways to achieve this at the application level. Hope this helps.</p>