I work as a software tester, so the majority of the code I write is designed to test other people's applications. I recently decided to check out the Hypothesis library for property-based unit testing out of curiosity. However, my code is often little more than a wrapper around the Requests library with some business logic to control the what and when of making requests, and there's very little to validate on my end. Shame, Hypothesis seems very cool, guess I'll just have to keep it in mind for future projects...

Hypothesis strategies as random input

Or not. Getting started with Hypothesis is as simple as decorating your test functions with the strategy you want. Why not use that in my system test scripts, so the inputs are provided to the REST API I'm checking? Totally contrived example, but let's say a /user endpoint accepts POST requests for updating a display name, and we want to quickly check a wide range of text values to see if anything falls over. Let's take a look:

import requests
from hypothesis import given
from hypothesis.strategies import text

# Do whatever auth setup etc you need here
session = requests.Session()
url = 'http://internalapp/user'

# Hypothesis 'given' decorator, and 'text' strategy
@given(text())
def update_name(displayname):
    payload = {'displayName': displayname}
    resp = session.post(url=url, data=payload)
    # Successful updates should always return 201 status codes
    # If anything else, show the JSON content.
    assert resp.status_code == 201, resp.json()
    # Assert the response contains the same text we sent.
    assert resp.json().get('displayName') == displayname, resp.json()

update_name()

Nothing too complicated here. Running this almost immediately finds a problem:

{"status": 400, "error": "Bad Request", 
"messages": {"displayName": "[displayName] cannot be left blank"} }

Ah. Empty strings return a bad request. That's OK, we expect the API to do that. We can tell hypothesis not to use empty strings by passing arguments to the strategy:

@given(text(min_size=1))

Running again reveals no problems. If you happened to be logging the inputs, you'd see it cycling through everything from the letter 'a' to what I'm pretty sure was Mandarin. All told, over a hundred inputs were sent at our REST API and all we had to do was use a single decorator, neato!

The joys of unexpected side effects

That was easy. So, new endpoint now, let's say /users. We can search with GET methods by providing parameters to this endpoint, so /users?firstName=John should give us a response like {"users": []} Ideally with the actual results for every user with John as their first name instead of an empty list, but we can worry about that later. Let's add that in below our previous code:

@given(text()):
def find_users(term):
    target = 'http://internalapp/users?firstName={term}'.format(term=term)
    resp = session.get(target)
    # Check the type of the 'users' value is a list
    assert isinstance(resp.json().get('users'), list), resp.json()

find_users()

Obviously this is a really lightweight test, we're not even checking for valid return values yet. However, it still managed to hit a problem:

{"status": 500, "error": "Internal Server Error", "path": "/users/",
"message": "DB Errors: codeOrType: Neo.ClientError.Statement.InvalidSyntax\nmessage: Invalid input \'0\': expected whitespace or a label name*SNIPPED REST OF ERROR*"}

...there's a Neo4J server behind this thing, and apparently there's a cypher injection risk if I send it anything but letters, judging by the 'InvalidSyntax' error coming back with a zero right in the query? Oops. This is clearly a lot more severe than whether the endpoint returns one John or five Freds for a given search.

Summary

Not too shabby - in less than a page of code, we've written a script that hits two endpoints with all kinds of weird input just to see what happens. The examples are pared down from what an automated system test suite is going to look like, but hopefully the concept is clear. Hypothesis is capable of a lot more than demonstrated here, I'm only scratching the surface with my own experiences with it so far. The Quick start guide provides more than enough to hit the ground running, enjoy!