search

Projectplace / pytest-tags

Tagging system for pytest tests
12 stars 7 forks source link

issue with pytest mark #8

Closed arautela-conga closed 4 years ago

arautela-conga commented 4 years ago

Hi, I am using pytest-tags annotated as below in the script. Upon execution I see that the correct number of items are getting collected but I do not see the results of all scripts in the allure report. I just wanted to get your inputs if this is an allure issue or pytest. Any inputs will be highly appreciated. Thank you in advance @pytest.mark.tags("smoke")

command used to execute script: pytest -s -v --tags smoke tests (my automated suite is under folder tests of the root directory)

Output of pip list from the virtual environment you are using allure-pytest 2.8.12 allure-python-commons 2.8.12 appdirs 1.4.3 atomicwrites 1.3.0 attrs 19.3.0 cached-property 1.5.1 cachetools 4.0.0 certifi 2019.11.28 chardet 3.0.4 client 0.0.1 colorama 0.4.3 comtypes 1.1.7 DateTime 4.3 defusedxml 0.6.0 delayed-assert 0.3.2 future 0.18.2 google-api-core 1.16.0 google-api-python-client 1.8.0 google-auth 1.11.3 google-auth-httplib2 0.0.3 googleapis-common-protos 1.51.0 httplib2 0.17.0 idna 2.9 isodate 0.6.0 jsonpath 0.82 lxml 4.5.0 more-itertools 8.2.0 packaging 20.3 pip 20.1.1 pluggy 0.13.1 properties 0.6.1 property 2.2 protobuf 3.11.3 py 1.8.1 pyad 0.5.15 pyasn1 0.4.8 pyasn1-modules 0.2.8 PyHamcrest 2.0.2 pyparsing 2.4.6 pytest 5.4.1 pytest-json 0.4.0 pytest-ordering 0.6 pytest-tags 1.7.2 pytz 2019.3 pywin32 227 pywinauto 0.6.8 requests 2.23.0 requests-toolbelt 0.9.1 rsa 4.0 selenium 3.141.0 setuptools 40.8.0 six 1.14.0 uritemplate 3.0.1 urllib3 1.25.8 wcwidth 0.1.9 zeep 3.4.0 zope.interface 5.1.0

Let say if 20 scripts are annotated as smoke. Upon execution, I see pytest collects all 20 out of the entire automated suite. But in the allure report I see results of only 10 scripts

biwwe-planview commented 4 years ago

I'm not sure to be honest, when I run the allure report it seems to pick them all up. Could it possibly be that the tests are in the same module and you need to expand them to see all tests when viewing your suites?

I ran pytest -s -rw --tags smoke --alluredir=%allure_result_folder% followed by allure serve %allure_result_folder%.

Pytest selected 23 items and allure reported 23 test cases, but only shows 11 entries when I look at my suites, as it's the 11 files that were touched, the actual tests are hidden until the files are expanded to show which tests were ran.

I'm not sure what the issue could be without more in-depth examples or screenshots.

arautela-conga commented 4 years ago

I'm not sure to be honest, when I run the allure report it seems to pick them all up. Could it possibly be that the tests are in the same module and you need to expand them to see all tests when viewing your suites?

I ran pytest -s -rw --tags smoke --alluredir=%allure_result_folder% followed by allure serve %allure_result_folder%.

Pytest selected 23 items and allure reported 23 test cases, but only shows 11 entries when I look at my suites, as it's the 11 files that were touched, the actual tests are hidden until the files are expanded to show which tests were ran.

I'm not sure what the issue could be without more in-depth examples or screenshots.

Thank you so much for your detailed reply. Appreciate it. I see I was missing --alluredir=%allure_result_folder% part after --tags 'tagname'. I created a dummy automation suite and did a sample run after making this change and it is working fine. Though I will soon try for the entire smoke suite to be fully sure. Expanding the module was not the issue here.

If you don't mind would you please direct me to documentation where I can find the meaning of tags viz. -s -rw (the one which you mentioned in the run command above after pytest). For some reason, I was not able to locate the documentation for the same

biwwe-planview commented 4 years ago

I haven't really found nice documentation of arguments besides running pytest -h. That'll show you all the arguments available to you based on stock arguments, and plugins, etc. Basically that and checking the code itself unfortunately, @arautela-conga

arautela-conga commented 4 years ago

I haven't really found nice documentation of arguments besides running pytest -h. That'll show you all the arguments available to you based on stock arguments, and plugins, etc. Basically that and checking the code itself unfortunately, @arautela-conga

This helps. Thank you so much :)