Integrate pytest-bdd and tavern with TestRail

There is a very handy plugin that integrates pytest with TestRail quite smoothly. It is called pytest-testrail and can be set up with minimal configuration. The only requirement is to create an API key for your TestRail account and pass the rest configurations as command line arguments or via a configuration file inside your automation project.

The problem

However, this plugin is meant to work only with a pure pytest project. Try using it with a framework built on top of pytest, like pytest-bdd or tavern and see it fail. This happens because the plugin is using custom pytest markers with parameters, which can only be applied inside python files (and not feature or yaml files).

The solution

The idea is to modify a traditional pytest marker in order to comply with the testrail plugin’s format. We should choose a format convention for the testrail markers which will embed the test case information, for example, testrail_C977.

To achieve this we are going to use the pytest_collection_modifyitems hook. This hook is called after all the pytest items (and their markers) have been collected. Every pytest marker that starts with the name testrail_ will be modified. The part after the underscore will be the test case id.

Putting the following code inside our conftest.py file will do the job:

    for item in items:
        test_cases = []
        for marker in item.iter_markers():
            if marker.name.startswith("testrail_"):
                test_case = marker.name.split("_")[1]
                if test_case not in test_cases:
                    test_cases.append(test_case)
        if test_cases:
            item.add_marker(pytest.mark.testrail(ids=tuple(test_cases)))

Now, we can simple declare a testrail marker like this:

pytest-bdd (feature file)

Feature: Add a new user

    @testrail_C979
    Scenario: Add a new user
        Given I am logged in as admin
        ...
        ...

Tavern (yaml file)

test_name: Add a new user
marks:
  - testrail_C977

stages:
  - name: Given I am logged in as admin
    ...
    ...

NOTE
In pytest-bdd you can declare the pytst markers using the plugin’s format inside the step files when you’re using the (single) scenario decorator.
However, I found this approach to be less flexible than using the markers directly inside the feature file.