I chose to use the PyTest testing framework, mainly
because of the minimal boilerplate code needed to write tests. Since
the Circular library is not a normal python library but a library which
is supposed to be used in the browser, testing it is not as
straightforward as a regular python library. Some parts of it
(e.g. the expression parser) can be tested as normal, however,
the parts dealing with the DOM (e.g. the templates) require some
preparation. The simplest approach (and the approach
I took first) would be to just write a dummy implementation of the Brython
browser module which would simulate the browser environment and use that
in the tests. This works quite well for basic tests, but it can only go so far.
After a comment
by Glyph on the Brython issues list,
I decided to try an alternative approach testing the library in a real
Browser environment.
Basic setup
The first problem I needed to overcome with this approach is to avoid
having to manually fire up the browser to run the tests. Since I wanted the
tests to run on Travis CI, this was a no-go.
Fortunately, this problem has long been solved.
The selenium browser automator even has a nice PyTest
plugin. This plugin
needs a driver (essentially a plugin for the specific browser used to run
the tests). I chose to use the PhantomJS driver
since it is headless and thus suitable for use on Travis CI.
Getting all the dependencies is a matter of a few straightforward
pip install and npm install commands:
/tmp/blog $ pip install pytest /tmp/blog $ pip install pytest-selenium /tmp/blog $ npm install phantomjs /tmp/blog $ export PATH=$PATH:/tmp/blog/node_modules/.bin/
The npm command installs the phantomjs binary into
node_modules/.bin and the last line puts it in the search path so that
that pytest-selenium can find it.
Writing a test using pytest-selenium is now very easy:
def test_google(selenium): selenium.get("http://www.google.com") logo = selenium.find_element_by_id("hplogo") assert logo.get_attribute('title') == "Google"
Lets save the above file as tests/test_google.py and run the test:
/tmp/blog $ mkdir tests/results /tmp/blog $ python -m pytest -rw --driver PhantomJS --html tests/results/results.html tests python -m pytest -rw --driver PhantomJS --html tests/results/results.html tests ======================================== test session starts ======================================== platform linux -- Python 3.4.3, pytest-3.0.2, py-1.4.31, pluggy-0.3.1 sensitiveurl: .* driver: PhantomJS rootdir: /tmp/blog, inifile: plugins: base-url-1.1.0, html-1.10.0, variables-1.4, selenium-1.3.1 collected 1 items tests/test_google.py . --------------------- generated html file: /tmp/blog/tests/results/results.html --------------------- ===================================== 1 passed in 3.21 seconds ======================================
The --html option will tell pytest to create a nice summary of the tests in tests/tesults/results.html.
Writing a real test
O.K., so the basic setup is ready and working. Now we will write a real test. Say we would like to test that the interpolation features of circular work, i.e. if we have a template
<span> Hello {{ name }} </span>
and bind it to a context with a suitable name attribute,
we would like to test that the result will be, e.g.,
<span> Hello Peter </span>
In reality the full page would look something like this:
<!DOCTYPE html> <html> <head> <!-- This tells Brython to look in the lib subdirectory of the web root for the circular library --> <link rel="pythonpath" href="lib" hreflang="py" /> <!-- This loads the Brython library (assuming it is installed in the lib subdirectory of the web root) --> <script src="/lib/brython/www/src/brython_dist.js"></script> <!-- The onLoadHandler calls the brython function to compile and run all python scripts on the page --> <script> var onLoadHandler = function() {brython();}; </script> <!-- This contains a python script which sets up the template and data binding --> <script type="text/python"> // Python code setting up the context, template and binding </script> </head> <body onload="onLoadHandler()"> <div id='template'> <span id='greeting'> Hello {{ name }} </span> </div> </body> </html>
where the script tag with the test/python type would contain:
from browser import document as doc, html from circular.template import Template, Context, set_prefix # Compile the template whose root element is # the div element with id 'template' tpl = Template(doc['template']) # Create a new context ctx = Context() # Bind the template to the context tpl.bind_ctx(ctx) # Set the name attribute to an appropriate value ctx.name = 'Peter' # This should update the template with the new value # (alternatively, we could just wait a few miliseconds # before the template updates automatically) tpl.update()
Lets save this to the file test_interpolation.html. If we opene the file in a
browser, it should greet Peter. Now we need to write a test for it. For this we
will start a web-server to serve the page and the libraries, say on localhost:7654.
This is easily done with the http module in Python 3 or SimpleHTTPServer
in Python 2:
/tmp/blog $ python -m http.server 7654
The test itself is pretty straightforward:
def test_interpolation(selenium): selenium.get("http://localhost:7654/test_interpolation.html") greeting = selenium.find_element_by_id("greeting") assert greeting.text == "Hello Peter"
However, this test would probably fail. The reason is that the assertion would be hit before the circular library had the time to do its job. We would like to start testing only after the setup completed. This is a common problem and selenium solves it using Waits. The idea is to wait for a certain element to appear on the page before continuing with the tests. We can then add this element at the end of our script:
from browser import document as doc, html from circular.template import Template, Context, set_prefix tpl = Template(doc['template']) ctx = Context() tpl.bind_ctx(ctx) ctx.name = 'Peter' tpl.update() # we add a div signalling that we # are ready to test doc <= html.DIV(id='finished')
In our test we use the WebDriverWait function:
from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC def test_interpolation(selenium): selenium.get("http://localhost:7654/test_interpolation.html") WebDriverWait(selenium, 2).until( EC.presence_of_element_located((By.ID, "finished")) ) greeting = selenium.find_element_by_id("greeting") assert greeting.text == "Hello Peter"
The parameter 2 tells selenium to wait at most 2 seconds and,
if the element is not present after the two seconds, raise an
exception (leading to a failed test).
Automating stuff
Now the test should work and we could call it a day. However though it works, it is a bit inconvenient. To write a test we need to create a html page with a lot of boilerplate code and a separate python test script with more boilerplate code. That is a lot of boring and unnecessary work. If writing tests is hard, no one will write them. Additionally, since the test is split into two files there is always the danger that when we do some changes, we update only one of them. It would be great if we could put everyting in a single file. Ideally, we would like to write the above test for example like this:
def pagescript_interpolation(): """ <span id='greeting'>Hello {{ name }}!</span> """ from browser import document as doc, html from circular.template import Template, Context, set_prefix set_prefix('tpl-') tpl = Template(doc['greeting']) ctx = Context() tpl.bind_ctx(ctx) ctx.name = 'Peter' tpl.update() def test_interpolation(selenium): wait_for_script(selenium) greeting = selenium.find_element_by_id("greeting") assert greeting.text == "Hello Peter!"
It turns out that, with a little bit of work, it is rather easy
to achieve this. The idea is to run a setup function before each
test which creates the html file for us putting the code of the
pagescript_interpolation function in the <head> tag
and putting the html code (which is specified in the docstring)
inside the <body> tag. With a clever use of the inspect
module and Jinja templates it is just a
few lines of code (saved in utils.py):
from jinja2 import Template tpl = Template(open('template.tpl', 'r').read()) def selenium_setup_helper(func): # Get the module from which the helper is called mod = inspect.getmodule(func) # The name of the script function is constructed # by replacing the "test_" prefix with the "pagescript_" # prefix script_name = 'pagescript_'+func.__name__[5:] # Get the actual script function from the module script = getattr(mod, script_name) # Get the code of the script function and add a command # to run it script_src = inspect.getsource(script)+"\n"+script_name+"()\n" # Get the content of the body tag from the docstring html = inspect.getdoc(script) # The file will be saved in the webroot subdirectory of the # directory where the setup_helper function is defined # (in practice, this should probably be somewhere more sensible) test_dir = os.path.dirname(inspect.getfile(selenium_setup_helper))+'/webroot/' # Construct the filename of the html file so that it includes # the name of the module and the name of the test out_file_name = mod.__name__+'-'+func.__name__[5:] # Open the file for writing out = open('%s/%s.html' % (test_dir, out_file_name), 'w') # Use the Jinja2 template to construct the html file out.write(tpl.render({ 'script': script_src, 'content': html })) out.close() def wait_for_script(selenium): # Find out the caller so that we can get the name # of the html page to open frame = inspect.stack()[1] mod = inspect.getmodule(frame[0]) fname = frame[3] test_html_file = mod.__name__ + '-' + fname[5:] # Open the test page selenium.get('http://localhost:7000/tests/%s.html' % test_html_file) # Wait for the finished element to appear on the page WebDriverWait(selenium, 2).until( EC.presence_of_element_located((By.ID, "finished")) )
The template.tpl contains:
<!DOCTYPE html> <html> <head> <link rel="pythonpath" href="lib" hreflang="py" /> <script src="../lib/brython/www/src/brython_dist.js"></script> <script> var onLoadHandler = function() {brython();}; </script> <script type="text/python"> from browser import document as doc, html {{ script }} doc <= html.DIV(id='finished') </script> </head> <body onload="onLoadHandler()"> {{ content }} </body> </html>
Now the test file looks almost exactly as we wanted:
from utils import wait_for_script, selenium_setup_helper def setup_function(func): selenium_setup_helper(func) def pagescript_interpolation(): """ <span id='greeting'>Hello {{ name }}!</span> """ from browser import document as doc, html from circular.template import Template, Context, set_prefix set_prefix('tpl-') tpl = Template(doc['greeting']) ctx = Context() tpl.bind_ctx(ctx) ctx.name = 'Peter' tpl.update() def test_interpolation(selenium): wait_for_script(selenium) greeting = selenium.find_element_by_id("greeting") assert greeting.text == "Hello Peter!"
Running the tests on Travis
Running the test on Travis CI is now a simple matter of writing the right
script. The .travis.yml file for the circular library
contains:
language: python python: - "3.4" cache: pip # command to install dependencies install: - pip install -r requirements.txt - npm install phantomjs # Tests script: ./management/test/run_tests.sh
and the script run_tests.sh contains
(approximately)
# The html files for the tests are located in the # ./tests/selenium/webroot folder. We start a # server to serve them on the localhost on port 7000 cd ./tests/selenium/webroot python -m http.server 7000 & cd ../../../ python -m pytest -rw --driver PhantomJS --ignore=./tests/selenium/webroot/ tests
Exercise
Write a simple Bottle app which would serve the test files and also allow testing AJAX requests, file uploads &c.