Element-34 / py.saunter

sample framework for using selenium with python and page objects
Apache License 2.0
94 stars 35 forks source link

--collectonly needs to show count of tests that would get executed #24

Closed mmaypumphrey closed 12 years ago

mmaypumphrey commented 12 years ago

Several of us have been making boo-boos w.r.t. test-counting lately, because of some -collectonly confusion. When one specifies -m with a valid argument and the --collectonly option, then sees a "collected items" message, one assumes that items match the collection parameters specified in pytest.ini AND the -m argument. Instead, the reflects just those specified in pytest.ini. Thus, in the example below, in order to determine how many "smoke" tests would get run during an actual execution run, one has to calculate 60-25. Is there some way to easily improve this usability issue? A couple suggestions:

  1. Annotate the "collected 60 items" to say something like "collected 60 items based on pytest.ini parameters."
  2. Output the actual count (35) somewhere in the output, clearly labeled.

adobe-MacBookPro:echosign mmaypump$ pysaunter.py -m smoke --collectonly ============================================================== test session starts ============================================================== platform darwin -- Python 2.7.2 -- pytest-2.2.0 collected 60 items . . . ------------------------ generated xml file: /Users/mmaypump/Desktop/qa/PYSAUNTER/echosign/logs/2012-01-11-18-48-51.xml ------------------------- ====================================================== 25 tests deselected by "-m 'smoke'" ====================================================== ========================================================= 25 deselected in 0.26 seconds ========================================================= adobe-MacBookPro:echosign mmaypump$

adamgoucher commented 12 years ago

This is a Py.Test-ism and not something that can be easily changed in Py.Saunter. Especially since it is in the runner itself and not just a plugin that can be stomped on.