danpovey / lilcom

Small compression utility
Other
33 stars 10 forks source link

Bug: Fresh installation is missing numpy #24

Closed janvainer closed 3 years ago

janvainer commented 3 years ago

Hi, when installing this package in a fresh virtual environment, the following error eccurs:

  Downloading https://files.pythonhosted.org/packages/8b/53/43f601d93e9f33da1c9d6909c1bf94390666726dbd2d18e76c377a5a1a9a/lilcom-1.1.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-szfvvzdg/lilcom/setup.py", line 26, in <module>
        import numpy
    ModuleNotFoundError: No module named 'numpy'

By installing numpy beforehand, it is then possible to install lilcom. Could this be mentioned in the README?

csukuangfj commented 3 years ago

It is mentioned in setup.py:

https://github.com/danpovey/lilcom/blob/7feb4b22084b0952b5b889a348b650d2685ffb51/setup.py#L59

It would be great if PyPI contains prebuilt wheel packages, so that pip install will install numpy automatically.

danpovey commented 3 years ago

OK.

On Sat, Nov 21, 2020 at 8:40 PM Fangjun Kuang notifications@github.com wrote:

It is mentioned in setup.py:

https://github.com/danpovey/lilcom/blob/7feb4b22084b0952b5b889a348b650d2685ffb51/setup.py#L59

It would be great if PyPI contains prebuilt wheel packages, so that pip install will install numpy automatically.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/danpovey/lilcom/issues/24#issuecomment-731573841, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO4HA67D64DXZB7IW4LSQ6YNFANCNFSM4T5YMMQA .

danpovey commented 3 years ago

Actually, can you LMK specifically what I have to do? Previously I was doing: python3 ./setup.py sdist; twine upload dist/* For a binary distribution I think I'd need to compile it on a variety of platforms (it contains C++ code).

On Sat, Nov 21, 2020 at 10:14 PM Daniel Povey dpovey@gmail.com wrote:

OK.

On Sat, Nov 21, 2020 at 8:40 PM Fangjun Kuang notifications@github.com wrote:

It is mentioned in setup.py:

https://github.com/danpovey/lilcom/blob/7feb4b22084b0952b5b889a348b650d2685ffb51/setup.py#L59

It would be great if PyPI contains prebuilt wheel packages, so that pip install will install numpy automatically.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/danpovey/lilcom/issues/24#issuecomment-731573841, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO4HA67D64DXZB7IW4LSQ6YNFANCNFSM4T5YMMQA .

csukuangfj commented 3 years ago

@danpovey There is something called manylinux. See https://github.com/pypa/manylinux

In summary: a wheel compiled for manylinux can run not only on Ubuntu, Debian, but also on other Linuxes, like CentOS, Redhat, etc. That is the meaning of many. This kind of wheel uses only a limited set of C APIs that are supported by GLIBC on many Linux OSes.

csukuangfj commented 3 years ago

manylinux is not used for k2, since k2 depends on PyTorch, but PyTorch does not provide a wheel for manylinux after version 1.5.0 or 1.3.0, I cannot remember the exact version.

csukuangfj commented 3 years ago

BTW: PyPI supports manylinux. So if you can build a manylinux wheel, it can be uploaded to PyPI so that others can use pip install lilcom to install it instead of compiling lilcom from source.

csukuangfj commented 3 years ago

I would like to help if you do not have time.

danpovey commented 3 years ago

It would be great if you could help. But I don't understand why pip can't know to install NumPy as a dependency of lilcom if lilcom is to be built from source? I don't see what's so difficult about that. Building from source will tend to give more efficient code, which might be a significant consideration.

On Sat, Nov 21, 2020 at 10:31 PM Fangjun Kuang notifications@github.com wrote:

I would like to help if you do not have time.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/danpovey/lilcom/issues/24#issuecomment-731587320, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO4SIZ6XBQ4WSUELQVTSQ7FMRANCNFSM4T5YMMQA .

csukuangfj commented 3 years ago

I just tried

pip install git+https://github.com/danpovey/lilcom

and got the same error message as the one posted by @janvainer.

I think the problem is that pip install will try to run setup.py, but setup.py assumes that numpy has already been installed.

danpovey commented 3 years ago

Maybe we need to create a requirements.txt?

On Sat, Nov 21, 2020 at 11:16 PM Fangjun Kuang notifications@github.com wrote:

I just tried

pip install git+https://github.com/danpovey/lilcom

and got the same error message as the one posted by @janvainer https://github.com/janvainer.

I think the problem is that pip install will try to run setup.py, but setup.py assumes that numpy has already been installed.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/danpovey/lilcom/issues/24#issuecomment-731592588, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO5764VZB74DEDUAQQDSQ7KWNANCNFSM4T5YMMQA .

csukuangfj commented 3 years ago

Maybe we need to create a requirements.txt?

I tried it locally once again with a requirements.txt inside the project but it does not help. It has the same error.

danpovey commented 3 years ago

I think the problem may be that the setup.py calls numpy.get_include(), and it imports numpy.

On Sat, Nov 21, 2020 at 11:39 PM Fangjun Kuang notifications@github.com wrote:

Maybe we need to create a requirements.txt?

I tried it locally once again with a requirements.txt inside the project but it does not help. It has the same error.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/danpovey/lilcom/issues/24#issuecomment-731595412, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLOZNZMEV72G5U3KFIG3SQ7NMJANCNFSM4T5YMMQA .

danpovey commented 3 years ago

See if you can find the answer from here https://stackoverflow.com/questions/54117786/add-numpy-get-include-argument-to-setuptools-without-preinstalled-numpy

On Sat, Nov 21, 2020 at 11:45 PM Daniel Povey dpovey@gmail.com wrote:

I think the problem may be that the setup.py calls numpy.get_include(), and it imports numpy.

On Sat, Nov 21, 2020 at 11:39 PM Fangjun Kuang notifications@github.com wrote:

Maybe we need to create a requirements.txt?

I tried it locally once again with a requirements.txt inside the project but it does not help. It has the same error.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/danpovey/lilcom/issues/24#issuecomment-731595412, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLOZNZMEV72G5U3KFIG3SQ7NMJANCNFSM4T5YMMQA .