search

pytest-dev / pytest-random-order

pytest plugin to randomise the order of tests with some control over the randomness
MIT License
65 stars 10 forks source link

Randomisation not compatible with custom test collection #26

Closed ringods closed 6 years ago

ringods commented 6 years ago

I have written two custom pytest plugins to test different aspects of an internal product. Each of these plugins has a custom way of collecting tests, passing custom subclasses of pytest.File, pytest.Collector and pytest.Item back to pytest.

The pytest-random-order only works with the global random bucket-type. All other bucket-type functions use the member module, which is defined in the pytest python plugin:

https://github.com/pytest-dev/pytest/blob/d61a7670a1d50eab367dfac03896d2811dbf5279/_pytest/python.py#L210

Since I pass back a tree with collections of tests (root -> File* -> Collector* -> Item*), it would be nice if randomisation could happen at the different levels of my custom pytest plugin.

jbasko commented 6 years ago

Is this what you are looking for? https://github.com/jbasko/pytest-random-order/commit/525a069a12c43e6734d773af0af1f4d641adbf13

ringods commented 6 years ago

@jbasko yes, I guess something like that. By looking at the implementation of parent1 and parent2, it's clear what you mean. But if someone would read this in the docs, would it be clear enough what it does? Maybe call it parent and grandparent?

ringods commented 6 years ago

@jbasko I noticed the 0.7.0 tag here on Github, but is there still something you need to do to have this version also available on Pypi? At the time of writing, the latest version is still 0.6.0 there.

https://pypi.org/project/pytest-random-order/#history

jbasko commented 6 years ago

I'm sorry, I forgot that for this project I need to manually do the upload to pypi.org. It is available now.