Satan's Test Case


It all started with an email that I can only paraphrase as the following:

Good morning,

Could you kindly review the unit tests for our project? We wish to verify that the our test cases fully cover the program’s logic.

~ Developer Friend

This sounds straightforward enough, right? No diabolical intention or winking emojis, so I responded:

Sure, I’ll review your tests and report back to you with what I find.

~ Me

The project included a small README document that had instructions on how to run the test suite. So far so good, this is something that every application should have.

I ran the testing command as instructed and was delightfully surprised to see the following output.

<OK> Configuring environment.
<OK> Starting tests.
<OK> 32 Tests run. 32 successes. 0 Failures.
<OK> Cleaning up.
<OK> Done.

It seemed like I was off to a good start so I opened up the file for the test case and started scratching my head.

When we design test cases it is not uncommon to “mock” various parts of the full application so that individual parts can be tested against what is assumed to be properly functioning code.

The test cases in this particular test suite were written against a mock application.

A mock that is, of the entire application.

I thought, no, I hoped, that I was wrong. Whoever had written the tests for this project had created fake test functions that each returned the expected value, and then asserted that it was true in the test case.

Why in the world something like this was written I have no clue, but it existed and I had to break the news to the owner of the application that they didn’t actually have any test coverage. Looking at the code for the application itself, I have two theories for why this mistake ended up being created.

  • First, the setup of the software was too tightly coupled. That is, the way it was written wasn’t modular enough to let parts of it be easily tested. Typically, if you have to mock large parts of your application during testing this is an indicator that you have bigger problems than you know.

  • And second, it was clear that whoever wrote the test cases for this code wasn’t the person who authored it. Through some extreme misunderstanding (of the principles of unit testing) I can only suspect that they failed to understand the design of the application itself and what the task assigned to them was intended to accomplish.

Test Case Anatomy

Enough about code problems, lets talk about what a good test case looks like.

Although there are numerous ways to write tests in every programming language, there are a few common ideas that most programming languages encourage. Under typical, prefered circumstances, test cases have the following layout.

  1. A file that contains one or more groups of tests.
  2. A class name or section title that groups a collection of individual tests cases.
  3. A series of cases that each test that a single function or functionality operates as expected.

Then, each test case has the following structure.

  1. A setup section that instantiates some prerequisite data.
  2. An execution section that runs the piece of code being tested.
  3. A verification section that checks that the produced result meets some kind of fixed constraint.

Also, as a general rule of thumb, if you open up a test case in your favorite text editor or IDE, and you have to scroll to see the entire test, then there is something wrong with either the tests or the software itself.

Here is an example of what a good test case should look like.

def test_fail_authentication_if_user_is_not_active(self):
    user = User.objects.create_user('foo', 'bar', 'baz')
    user.is_active = False
    user.save()
    self.model.objects.create(key='foobar_token', user=user)
    response = self.csrf_client.post(
        self.path, {'example': 'example'},
        HTTP_AUTHORIZATION=self.header_prefix + 'foobar_token'
    )
    assert response.status_code == status.HTTP_401_UNAUTHORIZED

Code snippet courtesy of Django REST Framework

Most software developers have either heard of, or encountered a set of unit tests in which the author placed every assertion within a single method. While this is annoying and is also poor practice, it could be much worse.

Broken Tail Python


“Broken tail” is a style of coding in Python that, while being syntactically valid, is visually awkward. Treat your code like art.

Broken tail continuations

MongoClient.database.get_or_create_object(name='cool_user', date='today',
				          text='example')

V.S

MongoClient.database.get_or_create_object(
    name='cool_user',
    date='today',
    text='example'
)

Observe the opening and closing parentheses around the method parameters. They are like little hands cradling the content. Each parameter has its own line which makes it easy for other developers to quickly scan the method signature.

Don’t do this:

short_name(
    "One parameter"
)

If it all fits on one line cleanly, do it.

short_name("One parameter")

Broken tail comments

This one is painful but I see it frequently, especially in cases when the comment line wouldn’t have exceeded 120 or even 80 characters (the line lengths commonly established in the PEP8 style guidelines for Python).

# This is a comment that must go onto the next
# line

V.S

# This is a comment that must go onto the next line

OR

# This is a comment that must
# go onto the next line

Readability is as important in programming as the function of the code itself.


Ultimately, Python isn’t the only programming language that is an accomplice of this coding style. Numerous other languages allow this to happen including Java Script, PHP, C, and so on. In the end, it isn’t necessarily something that the language itself should enforce, but rather a positive quality that the authors of code written in those languages should take into consideration when creating software.

This is a blog written by Gunther Cox, a random software developer who builds robots and strongly opposes the oxford comma. Robots built by Gunther Cox are likely to be proponents of the aforementioned stylistic punctuation and are likely to pitch a convincing argument for its use (to their creator's sheer and utter dismay).


❖ Other blogs by Gunther ❖