allure-framework / allure-python

Allure integrations for Python test frameworks
https://allurereport.org/
Apache License 2.0
718 stars 237 forks source link

AttributeError when using @allure.title("{param.value}") hides real errors in code #670

Closed bingroom closed 1 year ago

bingroom commented 2 years ago

With the usage provided in display_name.rst, I am going to use an AttrDict class to make the title of my tests more flexible.

As the following code shows, {_param.value} works fine to retrieve the value in fixture _param() according to the conditions.

import allure
import pytest

class AttrDict(dict):
    def __init__(self, *args, **kwargs):
        super(AttrDict, self).__init__(*args, **kwargs)
        self.__dict__ = self

@pytest.fixture(params=["a", "b"], scope="module")
def _param(request):
    # print(no_such_var)  #  <---- uncomment this may cause the issue
    if request.param == "a":
        value = 1
    else:
        value = 2
    return AttrDict({"value": value})

@allure.title("_param.value: {_param.value}")  # <---- Error occurred here when the fixture has error
def test_1(_param):
    print(_param.value)
    pass

But when an error occurred (e.g. print a non-defined variable in the fixture, as the above commented line in the fixture), this usage report an AttributeError: 'str' object has no attribute 'value' on the decorator statement rather than a NameError: name 'no_such_var' is not defined (i.e. my real error got hidden)

Full error messages are shown below, is there any solution or alternative to make me find out my real error but not the allure_name error ?

Screen Shot 2022-07-13 at 1 51 33 PM
_______________________________________________________ ERROR at setup of test_1[b] ________________________________________________________                            

self = <allure_pytest.listener.AllureListener object at 0x7f54aae6cf50>, item = <Function test_1[b]>                                                                    

    @pytest.hookimpl(hookwrapper=True)                                                                                                                                  
    def pytest_runtest_setup(self, item):                                                                                                                               
        if not self._cache.get(item.nodeid):                                                                                                                            
            uuid = self._cache.push(item.nodeid)                                                                                                                        
            test_result = TestResult(name=item.name, uuid=uuid, start=now(), stop=now())                                                                                
            self.allure_logger.schedule_test(uuid, test_result)                                                                                                         

        yield                                                                                                                                                           

        uuid = self._cache.get(item.nodeid)                                                                                                                             
        test_result = self.allure_logger.get_test(uuid)                                                                                                                 
        for fixturedef in _test_fixtures(item):                                                                                                                         
            group_uuid = self._cache.get(fixturedef)                                                                                                                    
            if not group_uuid:                                                                                                                                          
                group_uuid = self._cache.push(fixturedef)                                                                                                               
                group = TestResultContainer(uuid=group_uuid)                                                                                                            
                self.allure_logger.start_group(group_uuid, group)                                                                                                       
            self.allure_logger.update_group(group_uuid, children=uuid)                                                                                                  
        params = item.callspec.params if hasattr(item, 'callspec') else {}                                                                                              

>       test_result.name = allure_name(item, params)                                                                                                                    

fixturedef = <FixtureDef argname='_param' scope='module' baseid='test.py'>                                                                                              
group_uuid = '2b26de38-1624-4925-b73d-1b93225f0b95'                                                                                                                     
item       = <Function test_1[b]>                                                                                                                                       
params     = {'_param': 'b'}                                                                                                                                            
self       = <allure_pytest.listener.AllureListener object at 0x7f54aae6cf50>                                                                                           
test_result = TestResult(name='test_1[b]', status=None, statusDetails=None, stage=None, description=None, descriptionHtml=None, steps=[], attachments=[], parameters=[],
 start=1657688801629, stop=1657688801629, uuid='23374aca-c0ed-462b-b39b-b955174d8fa9', historyId=None, testCaseId=None, fullName=None, labels=[], links=[])             
uuid       = '23374aca-c0ed-462b-b39b-b955174d8fa9'  
../../.local/share/virtualenvs/YMUG9rwy/lib/python3.7/site-packages/allure_pytest/listener.py:88:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

item = <Function test_1[b]>, parameters = {'_param': 'b'}

    def allure_name(item, parameters):
        name = escape_name(item.name)
        title = allure_title(item)
>       return title.format(**{**parameters, **item.funcargs}) if title else name
E       AttributeError: 'str' object has no attribute 'value'

item       = <Function test_1[b]>
name       = 'test_1[b]'
parameters = {'_param': 'b'}
title      = '_param.value: {_param.value})'

../../.local/share/virtualenvs/YMUG9rwy/lib/python3.7/site-packages/allure_pytest/utils.py:114: AttributeError
========================================================= short test summary info ==========================================================
ERROR test.py::test_1[a] - AttributeError: 'str' object has no attribute 'value'
ERROR test.py::test_1[b] - AttributeError: 'str' object has no attribute 'value'
============================================================ 2 errors in 0.08s =============================================================

My environment: