Tables and numpy conflicting dependencies

[orpinchasov]

Doing a clean install of the package (python setup.py install) on Windows results in the following conflict:

error: numpy 1.18.5 is installed but numpy>=1.19.0 is required by {'tables'}

DeepLabCut-live specifically requires numpy < 1.19.0. Removing this requirements results in the following conflict (not sure though why it didn't install the latest numpy version):

error: numpy 1.19.0 is installed but numpy>=1.19.3 is required by {'opencv-python'}

I installed numpy==1.19.3 manually and got the setup.py working.

Finally, I'm getting the following errors when running dlc-live-test:

...
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
...
  File "d:\dev\DeepLabCut-live\.venv\lib\site-packages\tensorflow-2.8.0rc0-py3.9-win-amd64.egg\tensorflow\python\framework\dtypes.py", line 29, in <module>
    _np_bfloat16 = _pywrap_bfloat16.TF_bfloat16_type()
TypeError: Unable to convert function return value to a Python type! The signature was
        () -> handle

Not sure if it's related, but it does look suspicious because of the API version mismatch.

In any case if someone could run pip freeze and add its output to here, that'll be useful. Thanks

[MMathisLab]

Hey @orpinchasov !! Tables was just updated, so I bet the best thing to do is install tables3.6.1 (roll back), then the required NP version we need is set!

[sneakers-the-rat]

PR for this - https://github.com/DeepLabCut/DeepLabCut-live/pull/60

[orpinchasov]

I've been trying several combinations in the last couple of days, still didn't get it to work, even when I chose an older pytables version. I'm trying to work with the conda environment now, and then will try @sneakers-the-rat's version.

[orpinchasov]

Actually, the conda environment doesn't include the package, so there's nothing special there. I'm getting it to work as I'm using PyPI's version.

[sneakers-the-rat]

to disambiguate between what could be a compilation issue (tensorflow locally compiled against a different version of numpy so then swapping numpy versions would be incompatible even if when compiled again it would work) and an irresolvable version mismatch -- are you making fresh environments here or modifying an existing environment?

This combination posted in the PR works, subsetting the packages in question here

for both python==3.7.12 and python==3.9.9 on my local machine, and (3.7, 3.8, 3.8) * (windows, mac, ubuntu) on the ci the following work:

numpy                        1.18.5    NumPy is the fundamental package for array computing with Python.
opencv-python                4.5.5.62  Wrapper package for OpenCV python bindings.
tables                       3.6.1     Hierarchical datasets for Python
tensorflow                   2.7.0     TensorFlow is an open source machine learning framework for everyone.

poetry should fix the former specifications of just "tables", "opencv-python", and "tensorflow" without version constraints that are causing the problem.

[sneakers-the-rat]

so try making a virtual environment

mkdir ~/.venvs
python -m virtualenv ~/.venvs/dlclive
source ~/.venvs/dlclive/bin/activate

and then either try the poetry branch:

git clone https://github.com/DeepLabCut/DeepLabCut-live
# or git pull if you've already cloned
cd DeepLabCut-live
git switch poetry
pip install .

or here's a requirements.txt exported with poetry export --without-hashes > requirements.txt

requirements.txt

[sneakers-the-rat]

fixed by https://github.com/DeepLabCut/DeepLabCut-live/pull/60