search

pierrepo / PBxplore

A suite of tools to explore protein structures with Protein Blocks :snake:
https://pbxplore.readthedocs.org/en/latest/
MIT License
28 stars 17 forks source link

Remove test dir from PyPi package? #121

Closed pierrepo closed 8 years ago

pierrepo commented 8 years ago

Dear all, I've experienced very slow uploads of new releases of PBxplore to PyPi with my local internet connection. The final package is about 16 Mo on PyPi: https://pypi.python.org/pypi/pbxplore/1.3

In order to lighten the package size, could we consider removing the test directory (test scripts + test data) from the PyPi distribution system? I believe this test directory is made for development not production. In addition, the CI system ensures that PBxplore is 'fine' on most Python distributions. If needed, a user can 'test' PBxplore with data provided in the demo directory, as explained in the documentation. This will roughly reduce the size of the package by half.

@HubLot & @jbarnoud what do you think about this?

jbarnoud commented 8 years ago

I like that streamline users can test that everything work with their install. So I vote to keep the tests on pypi somehow. It does not have to be in the main pbxplore package, though. The way MDAnalysis does it is to have a separate package for the tests; we could do that. Note that it would require to tweak PBdata a bit (and have it work only if the tests are installed).

On 11-04-16 11:02, Pierre Poulain wrote:

Dear all, I've experienced very slow uploads of new releases of PBxplore to PyPi with my local internet connection. The final package is about 16 Mo on PyPi: https://pypi.python.org/pypi/pbxplore/1.3

In order to lighten the package size, could we consider removing the test directory (test scripts + test data) from the PyPi distribution system? I believe this test directory is made for development not production. In addition, the CI system ensures that PBxplore is 'fine' on most Python distributions. If needed, a user can 'test' PBxplore with data provided in the demo directory, as explained in the documentation. This will roughly reduce the size of the package by half.

@HubLot https://github.com/HubLot & @jbarnoud https://github.com/jbarnoud what do you think about this?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/pierrepo/PBxplore/issues/121

pierrepo commented 8 years ago

Good point! However, I believe you are more an 'advanced' user than a 'normal' one. Personally, I never tried to test packages installed through PyPi (maybe I should) and I'm pretty sure a lambda user won't either ;-)

HubLot commented 8 years ago

I like also having tests within the package... It's more convenient for user who wants to contribute. IMO, I don't really like the way of MDAnalysis separate things. I think it's understandable for a large package with many different tests on many different input files (like MDA) but for PBxplore, it's overkill and add too much complexity.

One reasonably solution could be reduce the size of the test & demo directories. XTC can be shrinked to a few Ko (instead of 13Mo right know in test_data). We could also use smaller PDB (190 chains in demo/beta3_IEGF12.pdb for 8Mo). We can easily decrease by 2-3 the size of the package with these tricks.

jbarnoud commented 8 years ago

Also, some files could be compressed.

On 11-04-16 15:49, Hub wrote:

I like also having tests within the package... It's more convenient for user who wants to contribute. IMO, I don't really like the way of MDAnalysis separate things. I think it's understandable for a large package with many different tests on many different input files (like MDA) but for PBxplore, it's overkill and add too much complexity.

One reasonably solution could be reduce the size of the test & demo directories. XTC can be shrinked to a few Ko (instead of 13Mo right know in |test_data|). We could also use smaller PDB (190 chains in |demo/beta3_IEGF12.pdb| for 8Mo). We can easily decrease by 2-3 the size of the package with these tricks

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/pierrepo/PBxplore/issues/121#issuecomment-208352768

pierrepo commented 8 years ago

Yes but I think you mix two different types of users:

That why I believe the test dir is useless in the PyPi package ;-)

jbarnoud commented 8 years ago

I would add a category: normal users that will not touch the code, but want to make sure everything is OK in their install as they will rely on the program for their paper.

On 11-04-16 19:21, Pierre Poulain wrote:

Yes but I think you mix two different types of users:

  • normal users who just expect PBxplore to work out of the box and who want to install PBxplore as smoothly and quickly as possible
  • advanced users who might like to improve PBxplore, the GitHub repo is then made for them.

That why I believe the test dir is useless in the PyPi package ;-)

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/pierrepo/PBxplore/issues/121#issuecomment-208457138

pierrepo commented 8 years ago

Agree but that is why we use a CI system and also why PyPi allows requirement.txt to have every dependence right.

Most of my published results are produced by software such Gromacs, PyMOL, Python, R, Numpy, Pandas... and I never performed a single test on these software, assuming they were working correctly (buy maybe I was wrong).

jbarnoud commented 8 years ago

Travis tells that it should be fine. It does not say that it works fine on your machine. You may use unexpected version of some dependencies for instance.

On 11-04-16 19:34, Pierre Poulain wrote:

Agree but that is why we use a CI system and also why PyPi allows requirement.txt to have every dependence right.

Most of my published results are produced by software such Gromacs, PyMOL, Python, R, Numpy, Pandas... and I never performed a single test on these software, assuming they were working correctly (buy maybe I was wrong).

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/pierrepo/PBxplore/issues/121#issuecomment-208465696

pierrepo commented 8 years ago

You're right about Travis. But to test that PBxplore works correctly, you need to:

HubLot commented 8 years ago

Okay fair enough.

pierrepo commented 8 years ago

Thanks

pierrepo commented 8 years ago

Hi guys,

I found in this thread that is was possible to have tests available in the package namespace (as Numpy does) : http://stackoverflow.com/questions/32390705/should-i-include-tests-and-pyc-files-when-building-package-with-setuptools

It would be great -- and a good way to keep tests in the PyPi package -- to do such a thing:

import pbxplore
pbxplore.test()

or

import pbxplore
pbxplore.test_regression()
pbxplore.test_functions()

with of course test_regression.py and test_functions.py still available to be run standalone as we do in the development process.

Do you think we could implement this? @jbarnoud / @HubLot do you like this?

pierrepo commented 8 years ago

@HubLot could you also shrink a bit test data as you proposed?

HubLot commented 8 years ago

fix with #122