18 December 2013

Factory Boy provides a convenient way to create valid and reusable objects for testing. It is an awesome replacement for Django’s brittle static fixtures. But what are the best practices for using this handy tool?

The subject under test should be spelled out in the test, otherwise you will be writing an Obscure Test.

This is bad:

from unittest import TestCase
from .factories import CampaignFactory

class PriceTests(TestCase):
    def test_share_price(self):
        campaign = CampaignFactory.build()
        self.assertEqual(10, campaign.share_price())

Looking at this test you cannot see why the share price should equal 10. If someone is looking at the test they shouldn’t  have to open up your factories.py file to see why 10 is the correct share price.

This is an example of the testing anti-pattern Mystery Guest.

The test reader is not able to see the cause and effect between fixture and verification logic because part of it is done outside the Test Method.

This is better:

from unittest import TestCase
from .factories import CampaignFactory

class PriceTests(TestCase):
    def test_share_price(self):
        campaign = CampaignFactory.build(funding_goal=100, available_shares=10)
        self.assertEqual(10, campaign.share_price())

Not only does the test reader not have to look in another file to understand the test, but if he wants to edit the factory to have a different funding goal he won’t break your test.

This does not mean that you need to go around spelling out all the relvent fields explicitly in every test.

This is okay:

def test_user_can_create_campaign(self):
    user = UserFactory(is_active=True)

But this is better:

def test_user_can_create_campaign(self):
    user = ActiveUserFactory()

Because ActiveUser encapsulates the important idea which is commonly used.

Why not just say?

def test_user_can_create_campaign(self):
    user = UserFactory()

And have the UserFactory produce an active user? After all, in most tests, except a test of the registration process, the user under test will be an active user.

Every model should have a factory with the same name as the model that has all the same defaults of the model.

So if:

User().is_active == False

is true, then:

UserFactory().is_active == False

Should also be true.

If you follow this practice for all of your factories it will be easier to reason about your tests. From these defaults you can easily subclass the base factory and make new factories like the ActiveUserFactory as shown below.

import factory

class UserFactory(factory.DjangoModelFactory):
    FACTORY_FOR  = User

    is_active = False

class ActiveUserFactory(UserFactory):
    is_active = True

Writing tests in this way can be really helpful when when new requirements come. Sometimes new requirements cause you to modify the meaning of a key concept in your system.

Maybe now all users must all sign the terms of service is order be considered active. If you are using the ActiveUserFactory in all the tests you can just go and change the defination of the ActiveUserFactory in one place, instead of changing it in every Selenium test with an active user (almost all of them).

Often the concept that is under test is more important than the actual attributes on the factory.

We have factories that encapsulate many of the key concepts on our site. As a crowdfunding platform we have many types of crowdfunding campaigns. Throughout the life of a campaign, it can be in many different states. These states change a campaign’s behavior on the site in a number of ways (which we of course need to test). These states are not all simply defined by one field on the Campaign model. For example a campaign is closed if it meets the definition below:

def is_closed(self):
    return ((self.is_approved_open_round is True) or
             and self.is_approved is True))
            and self.end_date
            and self.end_date <= datetime.now())

To this end, we have created a DraftCampaignFactory, PendingReviewCampaignFactory, LiveCampaignFactory, and ClosedCampaignFactory, so that we can easily refer to the key concepts without getting into the details, unless we are testing those details.

The level of detail of that you can see in the test setup should match the level of detail under test. Use this a guiding principle when writing your tests and your tests will be easier to maintain and will provide better documentation of your system.

blog comments powered by Disqus