The importance of maintaining tests

Today I read this paragraph from the Part 5 of the Django Tutorial, in the "When testing, more is better" section:

It might seem that our tests are growing out of control. At this rate there will soon be more code in our tests than in our application, and the repetition is unaesthetic, compared to the elegant conciseness of the rest of our code.

It doesn’t matter. Let them grow. For the most part, you can write a test once and then forget about it. It will continue performing its useful function as you continue to develop your program.

[...] At worst, as you continue developing, you might find that you have some tests that are now redundant. Even that’s not a problem; in testing redundancy is a good thing.

I strongly disagree with this assumption, specially because it is a text which is read by more self-taught programmers than software engineering professionals and it can be quite misleading. The code of tests is as important as production code, and therefore it should be maintained as well, at least if you want to gain the benefits of having a robust test suite. The reasoning for this statement is masterfully argued by Robert C. Martin in his book "Clean Code", and I will try to summarize it with my own words:

The change of test and production code is closely related, so you cannot expect modifying your production code without touching any test. However, a whole suite of "quick and dirty" tests could become a tangle structure that nobody wants to change. In that moment, the reliability of the test suite is gone and now it is another problem. Then you will find yourself in the situation of developing without tests: your production code becomes more rigid as it growths since you are not able to know if your broke something in your last change. Finally, the problem of a messy test suite has extended to your base code.

Of course, in a personal Django application it is not such a big deal, but don't forget that if this person becomes a member of a team to develop a bigger project, those habits should be corrected - and experience has taught me that it is easier to learn something the proper way than changing old and bad habits.

Installing mod_python in Apache2

mod_python is an Apache module to embed Python within your server. The steps to install it on Ubuntu and create a simple "Hello, world" page are the following ones:

Install mod_python

On Ubuntu, it is as simple as running:

$ sudo apt-get install libapache2-mod-python

Configure a new virtual host

I suggest to use a new virtual host instead of the default one. To do so, create a new file called test in the directory /etc/apache2/sites-available/ with the following content:

<VirtualHost *:80>
     ServerAdmin webmaster@localhost
     DocumentRoot /var/www/test/web/
     ErrorLog /var/www/test/logs/error.log
     CustomLog /var/www/test/logs/access.log combined

     <Directory /var/www/test/web/>
          Options Indexes FollowSymLinks MultiViews
          AllowOverride None
          Order allow,deny
          allow from all
          AddHandler mod_python .py
          PythonHandler mod_python.publisher
          PythonDebug On
     </Directory>
</VirtualHost>

And then create the directories /var/www/test/web/ and /var/www/test/web/logs. Now you can enable the new virtual host:

$ sudo a2dissite default
$ sudo service apache2 reload
$ sudo a2ensite test
$ sudo service apache2 restart

Create a "Hello, world" page

Create a file in /var/www/test/web/ with the name hello.py and write the following function:

def index(req):
    req.content_type = "text/html"
    return "<h1>Hello, world!</h1>"

Finally, open http://localhost/hello and it will show you a welcoming h1 element.

pyhue: A Python library for the Philips Hue personal lighting system

This week I have been developing a small Python module to work with the Philips hue system. This lighting system is formed by one or more LED lamps connected to a bridge, which is a network device that processes HTTP requests to change the state of the lamps.

For instance, let's suppose the IP address of your bridge is X.X.X.X and your username is test. If you want to set the hue of the lamp with ID 1 and its brightness to the maximum (255), you have to send a PUT request to hhtp://X.X.X.X/api/test/lights/1/state with the content {"hue":0, "bri":255}.

In Python, if you use an HTTPConnection object, it would be:

import json
try:
    from httplib import HTTPConnection  # Python 2.X
except ImportError:
    from http.client import HTTPConnection  # Python 3.X

content = json.dumps({"hue":0, "bri":255})
conn = HTTPConnection('X.X.X.X')
conn.request('PUT', '/api/test/lights/1/state', content)
response = conn.getresponse().read()
print(json.loads(response))

If the request is successfully sent, this prints the following response: [{"success":{"/lights/1/state/hue":0}}, {"success":{"/lights/1/state/bri":255}}] . This message confirms the state has been changed.

Wiht pyhue, I have tried to abstract this logic with an OOP approach: Instead of creating the HTTP requests manually, I have defined one class for each entity in the API, and I have overridden its __setattr__ methods to send the correct request apart from updating the corresponding attribute.

import pyhue

bridge = pyhue.Bridge('X.X.X.X', 'test')
light = bridge.get_light("1")
light.hue = 0
light.bri = 255

This is a basic example, but with this module you can also interact with groups and schedules. You can easily install pyhue with pip install pyhue and the source code is available on GitHub.

As a side note, I would like to mention that unittest has had an important role in this project. I have implemented a test suite that has allowed me to refactor easily without worrying about breaking anything. Thus, the amount of duplicated code is minimal and the length of the module is only around 150 lines, following also the guidelines of PEP8.

Disclaimer: Philips Hue is a product of Philips Lighting B.V. pyhue is an open source third party library and I am not associated with Philips in any way.

Compiling CPython on Fedora

This is the workflow I used to compile CPython and make a patch on a virtual machine with Fedora 17. The first steps may vary depending on the OS and the distribution, but the structure and the order of the steps is always the same.

---

First of all, you have to download Mercurial and obtain the appropiate headers:

$ sudo yum install mercurial
$ sudo yum-builddep python3

Then you can download the source code of CPython and compile it:

$ hg clone http://hg.python.org/cpython
$ cd cpython
$ ./configure --with-pydegub
$ make -s

Once this is done, run the tests, make your patch, and run the tests again to make sure your modification does not affect the functioning of CPython:

$ ./python -m test
$ (edit code)
$ hg diff > mypatch.patch
$ ./python -m test

If one test fails, you might want to run it again in verbose mode, so you can easily identify the origin of the problem. Let's say you want to rerun the test test_threading:

$ ./python -m test -v test_threading

Finally, and only if there was no errors while running your tests, you can submit your patch to the issue tracker.

Reference:

• Python Developer's Guide

Hello, world!

def hello_world():
    print("Hello, world!")

if __name__ == "__main__":
    hello_world()

In this blog I will post my progress through the Google Summer of Code 2013 (if I my application is accepted!), as well as some Python-related posts from my Spanish personal blog cajondeideas and my blog dedicated to PyNgine, a small 3D game engine I started developing some months ago.

My long-term goal with these blogs is to collect all this information in only one site, improve it as much as possible, and translate everything into English (I am not a native English speaker, so if I make any grammar mistake, please feel free to correct me in the comments below). I have some ideas about it, like using Django with Pygments and Markdown to implement the editor, but it will have to wait until the GSoC is over and I have some free time to implement it.


Thanks for reading,

Alex Rodas