KeyError in allure-pytest-bdd when executing test with --alluredir and using Examples in feature

SimonovaU commented 3 years ago

[//]: # I'm using pytest-6.2.4 for test automation with plugins allure-pytest-bdd-2.9.43, bdd-4.0.2. Python version is 3.6.9.

KeyError occurs when running pytest with flag --alluredir=reports.

I'm submitting a ...

[ ] bug report
[*] feature request
[ ] support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Test contains feature with Examples.

| empty_answer1    | <- variable name
| ---------------- |
| {"playlists":[]} | <- variable value
| {"playlists":[]} | <- variable value

Examples types have been explicitly declared in a python script via example_converters, as shown further:

scenarios('../features/requestroutePL.feature', example_converters=dict(locality=str,
                                                                        route=str,
                                                                        trip=str,
                                                                        mmetype=str,
                                                                        statusPL=str,
                                                                        urlwithroute=str,
                                                                        urlwithroute2=str,
                                                                        empty_answer1=str,
                                                                        urlwithroute3=str,
                                                                        empty_answer2=str))

As far as I get, console output says that the error happens during formatting:

    @pytest.mark.usefixtures(*function_args)
    def scenario_wrapper(request):
>       _execute_scenario(feature, scenario, request, encoding)

/home/yuliya/.local/lib/python3.6/site-packages/pytest_bdd/scenario.py:177: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/yuliya/.local/lib/python3.6/site-packages/pytest_bdd/scenario.py:143: in _execute_scenario
    _execute_step_function(request, scenario, step, step_func)
/home/yuliya/.local/lib/python3.6/site-packages/pytest_bdd/scenario.py:110: in _execute_step_function
    request.config.hook.pytest_bdd_before_step_call(**kw)
/home/yuliya/.local/lib/python3.6/site-packages/pluggy/hooks.py:286: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
/home/yuliya/.local/lib/python3.6/site-packages/pluggy/manager.py:93: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
/home/yuliya/.local/lib/python3.6/site-packages/pluggy/manager.py:87: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
/home/yuliya/.local/lib/python3.6/site-packages/allure_pytest_bdd/pytest_bdd_listener.py:65: in pytest_bdd_before_step_call
    step_result.name = get_step_name(request.node, step)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

node = <Function test_creating_playlists__responding_to_requests[AUTOTEST-9973991-00 -  0 (9973991\u0410)-\u0421\u0430\u043b\..._number=9973991&route_dir=0&route_type=\u0410&device_type=1&trip_short_name=00&boardNumber=773410446-{"playlists":[]}]>
step = <pytest_bdd.parser.Step object at 0x7f5db6a55198>

    def get_step_name(node, step):
        name = "{step_keyword} {step_name}".format(step_keyword=step.keyword, step_name=step.name)
        if hasattr(node, 'callspec'):
            for key, value in node.callspec.params.items():
                name = name.replace("<{key}>".format(key=key), "<{{{key}}}>".format(key=key))
>               name = name.format(**node.callspec.params)
E               KeyError: '"playlists"'

/home/yuliya/.local/lib/python3.6/site-packages/allure_pytest_bdd/utils.py:15: KeyError

What is the expected behavior?

Am I missing the point of Examples and example_converters or Example variables should be accepted in no relation with the symbols contained inside? Test passes if variable contains anything apart from braces ('{', '}'), otherwise it invariably fails.

What is the motivation / use case for changing the behavior?

I've tried to escape braces with '\', '%' symbols and making braces double as well. That was all in vain. The only thing that made test pass was to remove braces at all from the Examples. But this dubious step doesn't seem to be an answer. To top the whole thing off, this behavior is inconsistent, bcs the same expression in another test didn't raise any error.

Please tell us about your environment:

Allure version: 2.13.8
Test framework: pytest-6.2.4
Allure adaptor: -

Other information

https://stackoverflow.com/questions/68080885/keyerror-in-allure-pytest-bdd-when-executing-test-with-alluredir-and-using-exa

[//]: # ( . e.g. detailed explanation, stacktraces, related issues, suggestions . how to fix, links for us to have more context, eg. Stackoverflow, Gitter etc )

SimonovaU commented 3 years ago

There are the same problems with steps which have curl braces in their names. Nothing helped to extract variables from step (in step definition) either with the help of parsers.parse, parsers.re or custom parser provided in the official documentation (https://pytest-bdd.readthedocs.io/en/stable/). It seems that report can't be made bcs step name can't be formatted in a proper way.

olerudyshyn commented 1 year ago

I have faced the same issue for Scenario Outline and {} in table.

pytest==7.1.2 pytest-bdd==6.1.1 allure-pytest-bdd==2.11.1 allure-python-commons==2.11.1

allure-framework / allure-python