search

scalableminds / webknossos-wrap

Efficient 3D volume storage format for webKnossos and analyses
MIT License
8 stars 3 forks source link

Bundle lz4 with wkw lib #48

Closed normanrz closed 4 years ago

normanrz commented 4 years ago

Installing wkw via pip, currently requires lz4 to be installed on the system. It would be nice if wkw would just bundle the .so or .dll files for that.

amotta commented 4 years ago

That's already happening for Windows. Feel free to do the same for macOS.

On Linux that's less obvious to me. I thought that it's generally regarded poorly when applications / libraries ship with their own set of libraries (statically linked or hacked in via LD_LIBRARY_PATH).

How is this commonly solved in the Python ecosystem?

I agree that the situation is not handled very nicely right now. We should at least check for the presence of LZ4 and show a useful error message in case it's missing.

The fact that we bundle a DLL for Windows was mostly a hack, in my view. If we decide to "officially" bundle LZ4, then we should also add it to the build step of the continous integration pipeline, I think.

normanrz commented 4 years ago

I don't think there is a consistent solution for Python packages. Some packages ship all dependencies, so that the system is ready to run. No package would ship something basic as glibc, though. Some packages don't ship any dependency libs.

For this particular case, I think lz4 is a small-enough lib to be bundled. Also, I doubt that the perf-benefits from custom-compiled versions will be significant. On the upside, it would be very convenient to have it bundled, because then it just works (TM). I have forgotten to load the lz4 module numerous times :-D And it is easier for new users to get going.

Of course, we would add a build step for building lz4.