search

deeptools / pyBigWig

A python extension for quick access to bigWig and bigBed files
MIT License
212 stars 48 forks source link

[bug/feature] easier installation and better cross-platform compatibility #102

Closed shouldsee closed 4 years ago

shouldsee commented 4 years ago

I have encounter too many times where pip install pyBigWig failed me miserably. Is there any possible packaging magic that we can apply to keep pip install working? I am assuming conda is not a option.

shouldsee commented 4 years ago

pip install obviously does not handle any c-dependency like zlib.h / libcurl, which would throw bad errors to any potential users/newbies.

dpryan79 commented 4 years ago

pyBigWig should be able to handle libcurl being absent, but it absolutely requires zlib. There's nothing on the python side that I've ever seen to ease checking for dependencies like that, pip is quite simplistic. For most software, it's preferable to install using conda rather than pip since all dependencies can be handled for users. In this case, pyBigWig is available in the bioconda channel.

shouldsee commented 4 years ago

I just don't want to use conda. It would be really useful if you could indicate the c dependency at #installation in README.md so that any project depending on pyBigWig would be able to inherit that into their manpages.

pip does not seem to check c dependencies most of the time, which is a unfortunate shame. But I have a feeling there would be a way to fiddle with setuptools to inject some functions into pip install

This issue is also important because pyBigWig is acting as a python API which needs to be called within runtime and unlike other bash-oriented tools. For bash-based tools they can just be installed like a binary and packed into some docker image, but this isn't really feasible for a library that provides python API.

The reason I don't want to use conda is because it's too slow and lacking some online caching, unlike docker-based approaches where you can build and share images online. I have been using singularity with biocontainers to enhance the portability of workflows and they are much faster than conda, thought the building of these images usually involves some conda install.

dpryan79 commented 4 years ago

The majority of biocontainers are build artifacts from bioconda, they're just small docker containers with the conda package inside, so their runtime won't ever be faster than the conda packages (the same goes for the singularity containers automatically generated every night from them). I mention libcurl in the installation section, though I can make that more structured and add zlib.

shouldsee commented 4 years ago

Yes totally agreed. I am only referring to installation speed here and not execution speed, which is an important factor for developers who are creating and mutating pipelines to suit the experiment.

though I can make that more structured and add zlib.

It would be great if you could add a dependency section for that.

dpryan79 commented 4 years ago

Ah yes, the slowness of the conda solver is a constant source of annoyance for all of us :(

I'll update the readme and then close this once that's done.