search

nanobox-io / nanobox-engine-python

Engine for running Python apps on Nanobox
https://nanobox.io
MIT License
12 stars 13 forks source link

Python from pkgin is compiled with old flags #46

Closed bensalilijames closed 6 years ago

bensalilijames commented 6 years ago

When I was using this Nanobox engine, I was surprised to find that some of Python's popular packages such as tensorflow and torch would not install with pip:

$ pip install tensorflow
Could not find a version that satisfies the requirement tensorflow (from versions: )

It turned out that these packages only distribute wheels for Linux with the cp35m (or different Python version equivalent) ABI tag, but the version of Python that pkgin distributes can only run wheels with a cp35 or none ABI tag.

Sure enough, looking in the Makefile for building Python on pkgin, it comes with the --without-pymalloc flag, and so cannot support the above libraries out the box.

Nowadays, I believe most people compile their packages with pymalloc enabled - and indeed it is the default compiler option on recent versions of Python. Notably, the Python distributed by apt-get (and probably other package repos) is compiled with --with-pymalloc.

I can see a few suggestions to make building a wider breadth of packages work:

Thanks in advance - it would be great to have Nanobox able to run these out the box.

danhunsaker commented 6 years ago

We do use a different pkgin repository to begin with, so we can change settings on our end with ease. In fact, you could even submit the change yourself - https://github.com/nanobox-io/nanobox-pkgsrc-base - if you wanted. The tricky part is the delay while the new packages compile.

Very soon we'll be releasing Dockerfile-based deploys into beta, and those will let you use any OS and package combination you like, from anywhere in the Docker ecosystem. This new approach will let you switch to APT or RPM or even Portage at your own leisure.

bensalilijames commented 6 years ago

Thanks - that's great and very helpful. I'll have a look into pkgsrc-base and see if I can update the compiler flags in the Makefile there. Looking forward to Dockerfile-based deploys, as that's what we're currently doing!