Category Archives: Blog

Robert’s lighthearted and witty blog of all things technical.

Django Form Submission using jQuery and cookies to protect the csrf Token

I recently wrote about how to add csrf protection for jQuery Ajax submissions when using the ‘POST’ method.  We can use a similar method to protect forms while not exposing the csrf token.  When we set  ‘django.middleware.csrf.CsrfViewMiddleware’, in the MIDDLEWARE_CLASSES in ‘settings.py’, Django sets a csrf token as a cookie for each web page.  We can use the jQuery ‘.submit()’ method to read the cookie and add a hidden input field to the form prior to submission.  Django will process the form just as if you had included {% csrf_token %} in the form.  The benefit is that it is not easy to read the crsf token because it is stored in a cookie and browsers don’t expose cookies outside of the active page.

First, here is a code example of the simple but vulnerable way to add a csrf token to a Django form.

<form action="/" method="post">
    {% csrf_token %}
    {{ form.as_p }}
    <input type="submit" value="Category"/>
</form>

Now, we’ll change our form so that we can take advantage of the jQuery ‘.submit()’ method each time this form is submitted.  We are writing this using classes, so that we can write a site-wide ‘$(document).ready(function(){…}’ that takes action on these classes.  As a reminder, one can have more than one ‘$(document).ready(function(){…}’ function.  We use one in a javascript file that defines site-wide behaviors via classes and then one for each specific page that defines page specific behaviors.  Here is the updated form that does not expose the csrf token and makes use of site-wide classes.

<form action="/" method="post" class="YPForm">
    {{ form.as_p }}
    <input type="submit" value="Category" class="YPButton"/>
</form>

Now we can go to the site-wide javascript file and define the event handler that fires when forms of this class are submitted.  Please note that this code has a dependency on the ‘jquery.cookie.js’ library.  Also, please note that I am using C-style indentation to make everything fit onto this page.

function submitYPForm() {
    var csrfToken = "<input type='hidden' name='csrfmiddlewaretoken' 
                           value='" + $.cookie('csrftoken') + "' />"
    $(".YPButton").before(csrfToken);
}

$(document).ready(function(){
    // Enables setting csrf cookie into Ajax 'POST' header
    $.ajaxSetup({ beforeSend:setCookieInHeader });

    // Enables jQuery UI events (for CSS) on the buttons on the site
    $(".YPButton").button();

    // Submit event handler
    $(".YPForm").submit(submitYPForm);
});

I would recommend doing everything that really matters with https, but short of that, this is quite a bit more secure than exposing the the csrf token on your webpage.  Enjoy!

jQuery Ajax Cross Site Request Forgery Tokens with Python/Django

The Django tutorial demonstrates how to add a csrf token to a form, but how do we do it for Ajax?  The Django documentation explains how to do this. How does it work?  First, make sure that you set ‘django.middleware.csrf.CsrfViewMiddleware’, in your MIDDLEWARE_CLASSES in settings.py.  This will cause Django to set a csrf token as a cookie for each web page.  Other sites cannot access the cookie, but your page can.

One other thing stands out right away is this section.

In any template that uses a POST form, use the csrf_token tag inside the <form> element if the form is for an internal URL, e.g.:

<form action="." method="post">{% csrf_token %}

This should not be done for POST forms that target external URLs, since that would cause the CSRF token to be leaked, leading to a vulnerability.

We’ll get to proper form csrf protection, but first, let’s get it working for Ajax.  I took the code samples from the documentation and modified the code so that it will automatically get called for any Ajax submission.  I include this javascript file on any page that uses Ajax.  It is important to note that even though this file has a $(document).ready(function(){…}, you can include other javascript files with the same function.  They’ll all get called in order (assuming they download quickly enough).  Also, I am using C-style indentation so that my code sample will fit onto this page.

function csrfSafeMethod(method) {
    // these HTTP methods do not require CSRF protection
    return (/^(GET|HEAD|OPTIONS|TRACE)$/.test(method));
}

function sameOrigin(url) {
    // test that a given url is a same-origin URL
    // url could be relative or scheme relative or absolute
    var host = document.location.host; // host + port
    var protocol = document.location.protocol;
    var sr_origin = '/' + host;
    var origin = protocol + sr_origin;
    // Allow absolute or scheme relative URLs to same origin
    return 
        (url == origin || 
        url.slice(0, origin.length + 1) == origin + '/') ||
        (url == sr_origin || 
           url.slice(0, sr_origin.length + 1) == sr_origin + '/') ||
        // or any other URL that isn't scheme relative or 
        // absolute i.e relative.
        !(/^(\/\/|http:|https:).*/.test(url));
}

function setCookieInHeader (xhr, settings) {
   if (!csrfSafeMethod(settings.type) && sameOrigin(settings.url)) {
       // Send the token to same-origin, relative URLs only.
       // Send the token only if the method warrants CSRF protection
       // Using the CSRFToken value acquired earlier
       xhr.setRequestHeader("X-CSRFToken", $.cookie('csrftoken'));
   }
}

$(document).ready(function(){
    $.ajaxSetup({ beforeSend:setCookieInHeader });
});

(Update: You will need the jQuery cookie plugin for the $.cookie() function to work.)

Now that we’ve got csrf protection for Ajax submissions, we’ll have to work on proper form submissions next.

 

jQuery UI Autocomplete with Python/Django

I have been developing a website using Python/Django and the HTML 5 / Javascript / JQuery documentation is lacking.  I wanted to make a text widget that uses Ajax to query the database for suggestions, i.e. autocomplete.  After a lot of searching, here is what I figured out.  In short: JQuery UI.

I am using a class based Django form in my views.py to create the form with the autocomplete text entry on the index page of the site.  (I am going to have to do some C-style indentation to get this Python code to wrap on my page).

class IndexForm(forms.Form):
    category = forms.CharField(max_length=100,
          widget=forms.TextInput
             (attrs={'placeholder':'Category', 'autofocus':'true'}))

Here is the section from index.html that renders the form (again, I am wrapping lines to fit onto the blog page).

<section>
    <form action="/" method="post">
        {% csrf_token %}    <!-- <><><> FIXME -->
        {{ form.as_p }}
        <input type="submit" value="Category"/>
        {% if invalidCategory %}
            <p class ="YPBodyText">There is no match to 
                                       '{{ invalidCategory }}'.</p>    
        {% endif %}
    </form>
</section>

One thing to note is that I am using {% csrf_token %} directly on the webpage.  This is a no-no for front facing pages.  I have a work around for form submission and ajax and will post about that soon.  Please note that you must include jQuery and jQuery UI on your page for the autocomplete to work.  And now we’ll look at index.js for the jQuery UI call.  In our class based form in our views.py, we created a text widget called ‘category’.  It can be found in the DOM as ‘id_category’.

$(document).ready(function() {
    $("#id_category").autocomplete({ source:'SuggestCategory' });
});

As you can see, the autocomplete will make an ajax call to $URL/SuggestCategory.  Here is the line in our urls.py that routes the call to the right view in our views.py.

url(r'^SuggestCategory/$', views.SuggestCategory.as_view()),

And finally back to views.py for the class-based view that responds to the Ajax call.  Since the Ajax call has no side effects on the database, it is handled using the GET method.  I wrote an internal method that searches the ‘text’ column of the ‘category’ table in my objects.py.  If the start of any of these fields matches the submission in ‘starts_with’, we add it to a list, and returned it in JSON format.

class SuggestCategory(generic.View):

    def _get_category_list(self, max_results=0, starts_with=''):
        #import pdb; pdb.set_trace()
        categories = []
        cat_list = []
        if starts_with:
            categories = Categories.objects.filter
                           (category_text__istartswith=starts_with)

        max_len = len(categories)
        if max_results > 0 and max_len > max_results:
            max_len = max_results

        for cat in categories[:max_len]:
            cat_list.append(cat.category_text)

        return cat_list

    def get(self, request):
        #import pdb; pdb.set_trace()
        cat_list = self._get_category_list
                   (DEF_AUTO_COMPLETE_RESULTS, request.GET['term'])
        return HttpResponse(json.dumps(cat_list), 
                                 content_type = "application/json")

Enjoy!

PMI San Diego Chapter Lunch – February 13, 2014

Someone Has To Turn This Great Idea Into Dollars

I am presenting at the San Diego Chapter of the Project Management Institute’s monthly chapter lunch.

The ad is here: http://www.eventbee.com/event?eid=133713404

My presentation slides are available.  You are welcome to use them, but please credit greenroom if you do.  Before you download, please use a comment.

Robert Gordon PMI-SD Chapter Lunch 20140213

Tangible Results of Process Automation & Test Engineering

In the previous post, I described a production testing system where I was able to re-engineer an existing test setup. Along the way, we added a great number of automated efficiencies. So what were the results of this effort? The production line had been up and running three years before I got involved. In years 1, 2 & 3, the organization was able to ship almost exactly the same number of units per year. The problem was that the product was in high demand and units were stacking up awaiting test and calibration. Have a look at the graph below. In year four, I was able to implement the new automated process and ship the backlog of devices at a rate of 700% of the first the years’ shipments. Now in year five, we are on pace to exceed year four deliveries. The result? The organization cut prices by 20%. And the test technicians love the new setup. They hook the radios up for an overnight test and go home. In the morning, they update the test logs and go do other productive things for the rest of the day.

The_Lab_Setup_Units_Shipped_20130425

An Example of Process Automation and Test Engineering at Work

Update: Here is a discussion of the results of this effort.

Here is an example of where I was able to successfully apply process automation and test engineering to improve the throughput, effectiveness and profitability in a manufacturing process. The organization produces radios that require require quality control and calibration. The process in use prior to my involvement was very labor intensive and somewhat error prone. This is a theme that comes up again and again: Let the computers do the repetitive stuff to free up the humans to do the thinking stuff. This is why we make computers!

The solution involves quite a bit of software development. I wrote an application that

  • Controls the devices while under test.
  • Dynamically discovers the presence and type of environmental test chambers from among several dissimilar models.
  • Dynamically discovers the presence and type of spectrum analyzers.
  • Sets the temperature of the environmental test chambers at various points across an industrial temperature range (normally -30°C to +75°C).
  • Monitors and controls the environmental test chambers to ensure they are at the intended temperature.
  • Applies spectrum analyzer settings pertinent to a particular test step and takes measurements.
  • Calibrates and tests the radios at the various temperature set points. Calibration is handled via a measurement and adjustment feedback loop.
  • Dynamically detects a failing radio and automatically bypasses it to continue testing on remaining systems.
  • Creates Microsoft Excel formatted test reports.
  • Archives test reports and calibration tables to a central server.

So here is a picture of the setup:

img0

Here is a block diagram for the organization of the testing application. What you can see is a formed-based application with classes that represent the calibration as an object, the devices under test, the environmental test chambers and the spectrum analyzer. I implemented the device under test and environmental test chamber classes as “abstract base classes”. An abstract base class is used to enforce a standard interface which “child classes” must implement. This setup can dynamically discover the type of environmental test chamber and create the appropriate child class that can talk to that model of chamber while presenting a standard interface to the main form. I used the same design to allow the discovery of the specific model of device under test while presenting a standard interface to the calibration object.

img1

Next we have a flowchart of the overall test process.  Here is a description of the test steps.

  • Discover the environmental test chamber, devices under test and the spectrum analyzer.
  • Once all discovery is successful, enter a top-level loop to step through the temperature setpoints.
  • At each temperature setpoint, loop through each device under test to perform calibration and testing for each device.
  • Select device frequency and spectrum analyzer settings.
  • Calibrate each device.
  • When done with the current device move on to the next.
  • When done with all devices, move to the next frequency.
  • When done with all devices at all frequencies, move to the next temperature setpoint.
  • When done at all temperature setpoints, archive the test reports and calibration files.

Here is a flowchart of the test and calibration process.

img2

OK, now we are getting into the details.  Next is a flowchart of the environmental test chamber (often referred to as an “oven”) discovery process.  What is shown is an object oriented approach to process automation.  Prior to implementing this test application, the technician had to select the correct software version to match a specific oven.  This process is fully plug and play.

img3

And finally, another flowchart that describes the process to discover the devices under test.  This is also a plug and play setup where each device is detected when plugged into the test computer.  The test software probes the device to determine the specific model and creates the right object to communicate with the specific device type.

img4

We can do this for your business too.  We’ll even write the documentation!

Business Cards

My lovely Priska designed these beautiful cards.  She has a photography business.  Check out her site: Priska Photo.

You can click the images to see a larger view.

Here is the front side:

greenroom-BC-4

And the back:

greenroom-BC-verso2

Thank you sweetie!

Hello world!

Hello, world!

— More to follow soon!