search

termux-user-repository / pypi

PyPI index pages for Termux/Android
https://termux-user-repository.github.io/pypi/
4 stars 0 forks source link

include `pandas` and `torch` #2

Open knyipab opened 1 week ago

knyipab commented 1 week ago

Could we include the mentioned packages into tur pypi?

Actually, I want to package open-webui (a popular LLM frontend). It uses versioned packages which will easily conflict with general termux environment: https://github.com/open-webui/open-webui/blob/main/backend/requirements.txt

So my plan is to package it open-webui in a python virtual environment (perhaps venv) which is also the official suggested way.

licy183 commented 5 days ago

pandas is easy, but torch is difficult as it has many dependencies.

knyipab commented 5 days ago

I am not familiar and may be stupid to ask: Don't we just need to package torch wheel but not its dependencies? users can fall back to the official pypi repo for those deps installable with pip?

licy183 commented 5 days ago

Termux is a rolling distribution and libraries are not intended to be stable. It is almost not possible to use libraries from apt repo.

knyipab commented 5 days ago

Do you mean when pip update torch, it does not update the deps libs (cuz that needs pkg update && pkg upgrade)? So torch of newer version may end up using old deps from apt repo?

licy183 commented 5 days ago

No, quite the opposite. The dynamic library in whl may depend on the old version of the library in the apt repository.

knyipab commented 4 days ago

I see. So becuz users tend to not update pip packages (even tho they bump together with termux-packages & TUR), therefore the wheel should include all necessary deps libraries for it to function properly, am I right?

I suppose numpy should have the same issue provided its dependency on libopenblas. Unlike pypi, the numpy package on tur's pypi wheel file does not seem to contain a numpy.libs folder. So may i assume that numpy will use $PREFIX/lib/libopenblas.so and potentially suffer the same issue?

licy183 commented 4 days ago

Yes. Some of the wheels in TUR's pypi don't use auditwheel to package these libs. You can report it here and I'll rebuild them.