Azure / pytest-azurepipelines

Plugin for pytest that makes it simple to work with Azure Pipelines
MIT License
112 stars 35 forks source link

pytest marker ignored #51

Open bgatessucks opened 3 years ago

bgatessucks commented 3 years ago

I have several tests which need import xlwings; the package is not available on Linux machines so I mark them with needsxw. In azure-pipelines.xml I have:

- script: |
    pip install pytest pytest-cov pytest-azurepipelines
    python -m pytest -m "not needsxw" --cov=. --cov-report=xml --cov-report=html
  displayName: 'Test with pytest'

however the build still fails. How can I have Azure ignore the tests with a given marker ?

Example of failing test:

@pytest.mark.needsxw
class TestToy(TestCase):

Output:

2021-04-14T09:53:02.0497514Z ##[section]Starting: Test with pytest
2021-04-14T09:53:02.0651740Z ==============================================================================
2021-04-14T09:53:02.0652070Z Task         : Command line
2021-04-14T09:53:02.0652783Z Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2021-04-14T09:53:02.0653055Z Version      : 2.182.0
2021-04-14T09:53:02.0653256Z Author       : Microsoft Corporation
2021-04-14T09:53:02.0653556Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2021-04-14T09:53:02.0653921Z ==============================================================================
2021-04-14T09:53:04.9307495Z Generating script.
2021-04-14T09:53:04.9319416Z ========================== Starting Command Output ===========================
2021-04-14T09:53:04.9320989Z ##[command]"C:\Windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "D:\a\_temp\13ef6923-1927-47a7-beea-7dfce92d8600.cmd""
2021-04-14T09:53:05.3509712Z Requirement already satisfied: pytest in c:\hostedtoolcache\windows\python\3.8.9\x64\lib\site-packages (6.2.3)
2021-04-14T09:53:05.4922285Z Collecting pytest-cov
2021-04-14T09:53:05.5825788Z   Downloading pytest_cov-2.11.1-py2.py3-none-any.whl (20 kB)
2021-04-14T09:53:05.6711325Z Collecting pytest-azurepipelines
2021-04-14T09:53:05.6868153Z   Downloading pytest_azurepipelines-0.8.0-py3-none-any.whl (6.2 kB)
2021-04-14T09:53:05.7670785Z Requirement already satisfied: packaging in c:\hostedtoolcache\windows\python\3.8.9\x64\lib\site-packages (from pytest) (20.9)
2021-04-14T09:53:05.7679576Z Requirement already satisfied: iniconfig in c:\hostedtoolcache\windows\python\3.8.9\x64\lib\site-packages (from pytest) (1.1.1)
2021-04-14T09:53:05.7687472Z Requirement already satisfied: attrs>=19.2.0 in c:\hostedtoolcache\windows\python\3.8.9\x64\lib\site-packages (from pytest) (20.3.0)
2021-04-14T09:53:05.7695046Z Requirement already satisfied: py>=1.8.2 in c:\hostedtoolcache\windows\python\3.8.9\x64\lib\site-packages (from pytest) (1.10.0)
2021-04-14T09:53:05.7700729Z Requirement already satisfied: toml in c:\hostedtoolcache\windows\python\3.8.9\x64\lib\site-packages (from pytest) (0.10.2)
2021-04-14T09:53:05.7713583Z Requirement already satisfied: atomicwrites>=1.0 in c:\hostedtoolcache\windows\python\3.8.9\x64\lib\site-packages (from pytest) (1.4.0)
2021-04-14T09:53:05.7723018Z Requirement already satisfied: pluggy<1.0.0a1,>=0.12 in c:\hostedtoolcache\windows\python\3.8.9\x64\lib\site-packages (from pytest) (0.13.1)
2021-04-14T09:53:05.7730651Z Requirement already satisfied: colorama in c:\hostedtoolcache\windows\python\3.8.9\x64\lib\site-packages (from pytest) (0.4.4)
2021-04-14T09:53:06.3306437Z Collecting coverage>=5.2.1
2021-04-14T09:53:06.3465580Z   Downloading coverage-5.5-cp38-cp38-win_amd64.whl (211 kB)
2021-04-14T09:53:06.4224833Z Requirement already satisfied: pyparsing>=2.0.2 in c:\hostedtoolcache\windows\python\3.8.9\x64\lib\site-packages (from packaging->pytest) (2.4.7)
2021-04-14T09:53:06.8486706Z Installing collected packages: coverage, pytest-cov, pytest-azurepipelines
2021-04-14T09:53:07.3072825Z Successfully installed coverage-5.5 pytest-azurepipelines-0.8.0 pytest-cov-2.11.1
2021-04-14T09:53:15.4936089Z ============================= test session starts =============================
2021-04-14T09:53:15.4936932Z platform win32 -- Python 3.8.9, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
2021-04-14T09:53:15.4937606Z rootdir: D:\a\1\s, configfile: pytest.ini
2021-04-14T09:53:15.4938057Z plugins: anyio-2.2.0, azurepipelines-0.8.0, cov-2.11.1
2021-04-14T09:53:15.4938518Z collected 22 items / 3 errors / 19 selected
2021-04-14T09:53:15.4971019Z ##[warning]invalid escape sequence \ 
2021-04-14T09:53:15.4981687Z ##[warning]invalid escape sequence \ 
2021-04-14T09:53:15.4982809Z ##[warning]invalid escape sequence \ 
2021-04-14T09:53:15.4983786Z ##[warning]invalid escape sequence \ 
2021-04-14T09:53:16.9908013Z Result Attachments will be stored in LogStore
2021-04-14T09:53:17.0182557Z Run Attachments will be stored in LogStore
2021-04-14T09:53:17.0737439Z ##[error]3 test(s) failed, 22 test(s) collected.
2021-04-14T09:53:17.0768999Z Reading code coverage summary from 'D:\a\1\s\coverage.xml'
2021-04-14T09:53:17.2006345Z 
2021-04-14T09:53:17.2006796Z =================================== ERRORS ====================================
2021-04-14T09:53:17.2012753Z ________________ ERROR collecting mrlib/toy/tests/test_toy.py _________________
2021-04-14T09:53:17.2013213Z ImportError while importing test module 'D:\a\1\s\mrlib\toy\tests\test_toy.py'.
2021-04-14T09:53:17.2013693Z Hint: make sure your test modules/packages have valid Python names.
2021-04-14T09:53:17.2014195Z Traceback:
2021-04-14T09:53:17.2014905Z C:\hostedtoolcache\windows\Python\3.8.9\x64\lib\importlib\__init__.py:127: in import_module
2021-04-14T09:53:17.2015601Z     return _bootstrap._gcd_import(name[level:], package, level)
2021-04-14T09:53:17.2015998Z mrlib\toy\tests\test_toy.py:4: in <module>
2021-04-14T09:53:17.2016348Z     from mrlib.toy.toy import make_double, make_total
2021-04-14T09:53:17.2017103Z mrlib\toy\toy.py:2: in <module>
2021-04-14T09:53:17.2017582Z     import xlwings as xw
2021-04-14T09:53:17.2017930Z E   ModuleNotFoundError: No module named 'xlwings'
ferrouswheel commented 2 years ago

This is an error during test collection, not an error when the test is being run.

Marks won't help ignore this issue. If you can't fix the error, then you may need to explore ways of filtering which files tests are collected from.