switching to not enabled by default.

[stas00]

To "The plugin is enabled by default" - would it be an unreasonable request to change the module not to be enabled by default and use a flag to enable it instead? Perhaps an env variable could be used to enable it globally w/o a flag. I'd like to start using this module in our environment, but its default behavior is not suitable for everybody.

Alternatively, do you know of any other similar modules that don't enable themselves by default? The only other I found of the same quality is pytest-randomly which has a similar issue.

I'm not questioning your choice in the matter of implementing it this way, just asking if perhaps it can be made more flexible so that it'll be accessible to more people.

Thank you.

[jbasko]

So far I have been referring users to https://docs.pytest.org/en/latest/plugins.html#deactivating-unregistering-a-plugin-by-name but I just tried to then enable the plugin on request with no success so it looks like a real problem.

I will investigate more but if there is indeed no nice way to enable the plugin on request if its explicitly disabled then I'll have to change this.

[stas00]

Thank you for looking into it, @jbasko

I believe this is a problem for anybody who may want to put pytest-random-order in their project dependencies, because now if users do pip install myproject it'll change their system's behavior w/o them realizing that. Even if I restrict the dependency to the developer section of config.py/requirements.yml, it'll still be a not always welcome surprise.

And yes, I realize, I don't have to put it in dependencies, but it'd be useful to do so, instead of adding instructions like "if you want a good way to randomize the tests, install this module, but then don't forget to run 'something-disable-the-default' to change its default behavior" since people will miss/forget to do so and then there will be complaints about how could you do something like that to my system. I hope I explained it clearly.

[jbasko]

Looks like I hadn't done a good job in studying the plugin registration. Re-enabling works, but the name to be used is a bit cumbersome.

So, if in your pytest.ini or setup.cfg or other pytest config you add line addopts = -p no:random-order then you can re-enable the plugin (for now, v0.0.8) with pytest -p pytest_random_order.plugin

[stas00]

This is good, but it must not be enabled by default on install. I can't possibly tell users that after installing this module for the rest of that environment's existence they have to edit setup.cfg of any other project they may run tests in to disable it. Does it make sense? It needs to be enabled with a flag (any will do) but not to be enabled by default on install to make any project to depend on it.

What I suggested is that if other people want to have it enabled by default, there is an easy workaround: put export PYTEST_ALWAYS_RANDOM=1 in their .bashrc (and of course this plugin will need to check that the env var is set to 1')

Perhaps my communication is not clear - just to check that we are on the same page - you realize that if the dependencies of my project install this module for other users, it will be breaking their environment without their permission.

[jbasko]

I understand. For now all I can suggest is use the workaround. When I release the next version it will probably be disabled by default.

The traditional workflow is to have a separate virtualenv for each project and a separate test configuration for each project (setup.cfg , pytest.ini, etc.) so the defaults of installed dependencies don’t really matter.

A plugin that is disabled is not loaded and cannot add any command line arguments. Plugins are enabled in a standard fashion with that -p flag. The problem with this plugin is that importing its package doesn’t register hooks. That will definitely be fixed on the next release.

This is a pytest plugin and not a separate tool so it will try to follow along the lines of pytest plugins.

Thank you for your feedback.

[jbasko]

Also, pytest or its plugins shouldn’t be in your end product’s dependencies. It’s a development dependency.

If your end product includes pytest then it surely must configure it with whatever defaults make most sense for your product and allow users to override it.

[stas00]

Yes, it's a dev dependency. I can't control how other developers use their virtual envs. But they object to me introducing this dependency as long as it's enabled by default. And also we require contributors to install developer dependencies so that they can validate their PRs, so the affected group is pretty large.

I have never meant to suggest not to enable the plugin, I suggested not to enable its randomization w/o a flag, but I didn't know enough about pytests workings to phrase that correctly. Now that you explained the specifics of pytest's framework, I can re-word my request to:

Please don't enable the randomization by default (while enabling the plugin by default), and let the users use a flag to enable the randomization.

Is my request more sensible now?