Issues with Python dependencies on MacBook M1 Pro

[bufo24]

I get several errors when I try to run the 0_setup.sh on my machine.

M1 Pro with 16GB Python version: 3.9.7

ERROR: Failed building wheel for numpy Failed to build lumpy ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects ERROR: Could not find a version that satisfies the requirement opencv-contrib-python-headless==4.4.0.46 (from versions: 3.4.11.45, 3.4.13.47, 3.4.14.51, 3.4.15.55, 3.4.16.57, 4.4.0.46, 4.5.1.48, 4.5.2.52, 4.5.3.56, 4.5.4.58) ERROR: No matching distribution found for opencv-contrib-python-headless==4.4.0.46

[chychkan]

@bufo24 thanks a lot for reporting! Looking into it.

[chychkan]

Hey @bufo24! I pushed a fix just now. Could you please confirm if it works for you now?

[rdlauer]

Just playing around with this on an M1 Mac, I still get the following error (looks like somewhere on the chain Python 3.10 is being used, even though my python3 version is 3.9.7):

Using requirements.txt for Python 3.10.0
Collecting numpy==1.19.5
  Using cached numpy-1.19.5.zip (7.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
ERROR: Could not find a version that satisfies the requirement opencv-contrib-python-headless==4.1.2.30 (from versions: 3.4.11.45, 3.4.13.47, 3.4.14.51, 3.4.15.55, 3.4.16.57, 4.4.0.46, 4.5.1.48, 4.5.2.52, 4.5.3.56, 4.5.4.58)
ERROR: No matching distribution found for opencv-contrib-python-headless==4.1.2.30

[chychkan]

@rdlauer I would suggest to try deleting the temp directory .dfl that is created by the scripts as it contains the virtual env. If you attempted to run this on Python 3.10 before, then the virtual env may still contain Python 3.10.

[rdlauer]

I'm a little confused as it looks like .dfl is created as part of the setup script? This error is on the initial run of that script (so there is no .dfl directory to delete). Not sure if this will tell you anything:

./scripts/0_setup.sh 
Cloning DeepFaceLab
Cloning into '.dfl/DeepFaceLab'...
remote: Enumerating objects: 247, done.
remote: Counting objects: 100% (247/247), done.
remote: Compressing objects: 100% (222/222), done.
remote: Total 247 (delta 24), reused 142 (delta 16), pack-reused 0
Receiving objects: 100% (247/247), 253.22 MiB | 25.20 MiB/s, done.
Resolving deltas: 100% (24/24), done.
Updating files: 100% (211/211), done.
created virtual environment CPython3.10.0.final.0-64 in 153ms
  creator CPython3Posix(dest=/Users/me/Documents/DeepFaceLab_MacOS/.dfl/env, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/me/Library/Application Support/virtualenv)
    added seed packages: pip==21.3.1, setuptools==58.3.0, wheel==0.37.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
Using requirements.txt for Python 3.10.0
Collecting numpy==1.19.5
  Using cached numpy-1.19.5.zip (7.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
ERROR: Could not find a version that satisfies the requirement opencv-contrib-python-headless==4.1.2.30 (from versions: 3.4.11.45, 3.4.13.47, 3.4.14.51, 3.4.15.55, 3.4.16.57, 4.4.0.46, 4.5.1.48, 4.5.2.52, 4.5.3.56, 4.5.4.58)
ERROR: No matching distribution found for opencv-contrib-python-headless==4.1.2.30

I guess it's this line that stands out to me: created virtual environment CPython3.10.0.final.0-64

[chychkan]

@rdlauer Yes, indeed, the issue is that Python 3.10 in your env is still picked up as the active one. I would recommend using https://github.com/pyenv/pyenv. It helps with managing multiple Python versions. Once you have it installed, you can install Python 3.9.7 with command:

$ pyenv install 3.9.7

and then make it active with command:

$ pyenv global 3.9.7

Basically, for these scripts to work you need to make sure that in your shell command python3 -V gives Python 3.9.7. Hopefully, this is helpful.

[rdlauer]

I'll try that, but right now I only have 3.9.7 installed:

python3 -V
Python 3.9.7

[chychkan]

@rdlauer Could you please also share what virtualenv --version returns?

[rdlauer]

Interesting:

virtualenv 20.10.0 from /opt/homebrew/Cellar/virtualenv/20.10.0/libexec/lib/python3.10/site-packages/virtualenv/__init__.py

[chychkan]

Ok, looks like that's the problem. The pyenv should help with that.

[rdlauer]

I appreciate the tips! Yeah this is strange, even after trying pyenv I see output from the setup scriptUsing requirements.txt for Python 3.10.0.

[chychkan]

Hey @rdlauer! Recently, I upgraded my Mac to OS X 12 and installed Python 3.10 and I started having the same issue. And I was able to fix it by adding this line to the end of my ~/.zshrc (add it to your .bash_profile if you're using Bash):

export PATH="$HOME/.pyenv/shims:$PATH"

And make sure you restart your shell.

After that on my Mac everything works as expected:

~ pyenv global 3.8.6
~ python -V
Python 3.8.6
~ virtualenv --version
virtualenv 20.4.2 from /Users/alex/.pyenv/versions/3.8.6/lib/python3.8/site-packages/virtualenv/__init__.py
~ pyenv global 3.10.0 
~ python -V           
Python 3.10.0
~ virtualenv --version
virtualenv 20.10.0 from /Users/alex/.pyenv/versions/3.10.0/lib/python3.10/site-packages/virtualenv/__init__.py

[rdlauer]

Thanks - but this is where my lack of Python knowledge is killing me :)

~ pyenv global 3.9.7
~ python -V
Python 3.9.7
~ virtualenv --version
virtualenv 20.10.0 from /opt/homebrew/Cellar/virtualenv/20.10.0/libexec/lib/python3.10/site-packages/virtualenv/__init__.py

I wonder if it's because homebrew is managing my virtualenv installation?

[chychkan]

@rdlauer I don't think it is because of homebrew. You need to make sure the $HOME/.pyenv/shims is at the beginning of your PATH. Could you please try this sequence of commands and share the output:

pyenv global 3.9.7
echo "$PATH"
export PATH="$HOME/.pyenv/shims:$PATH"
echo "$PATH"
python -V
virtualenv --version

[rdlauer]

Happy to:

~ pyenv global 3.9.7
~ echo "$PATH"
/Users/roblauer/.pyenv/shims:/opt/homebrew/bin:/usr/local/bin:/Users/roblauer/.nvm/versions/node/v14.15.5/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/Users/roblauer/Library/Android/sdk/platform-tools:/Users/roblauer/Library/Android/sdk/tools:/Users/roblauer/Library/Android/sdk/tools/bin:/Users/roblauer/Library/Android/sdk/emulator
~ export PATH="$HOME/.pyenv/shims:$PATH"
~ echo "$PATH"
/Users/roblauer/.pyenv/shims:/Users/roblauer/.pyenv/shims:/opt/homebrew/bin:/usr/local/bin:/Users/roblauer/.nvm/versions/node/v14.15.5/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/Users/roblauer/Library/Android/sdk/platform-tools:/Users/roblauer/Library/Android/sdk/tools:/Users/roblauer/Library/Android/sdk/tools/bin:/Users/roblauer/Library/Android/sdk/emulator
~ python -V
Python 3.9.7
~ virtualenv --version
virtualenv 20.10.0 from /opt/homebrew/Cellar/virtualenv/20.10.0/libexec/lib/python3.10/site-packages/virtualenv/__init__.py

[chychkan]

@rdlauer Ok, one more thing. Try this:

pyenv exec pip install virtualenv
virtualenv --version

[rdlauer]

Same virtualenv version is returned:

virtualenv 20.10.0 from /opt/homebrew/Cellar/virtualenv/20.10.0/libexec/lib/python3.10/site-packages/virtualenv/__init__.py

Should I be targeting virtualenv 20.4.2 when I install it? Something like: pyenv exec pip install virtualenv==20.4.2?

[chychkan]

I don't think it is related to a specific version of virtualenv, but rather which python version it belongs to. In either case, I would suggest you to report this issue and ask for advice in https://github.com/pyenv/pyenv (or maybe it has already been discussed and solved there). The issue is not caused by scripts in this repo.

[github-actions[bot]]

The issue is tagged as "stale issue" due to no activity in 30 days. If there is no activity for 15 more days, the issue will be closed.

[gimmel]

I tried all the hints from above and downgraded my python version from 3.10 to 3.9.7 with pyenv.

python3 --version
Python 3.9.7

virtualenv --version
virtualenv 20.13.0 from /opt/homebrew/Cellar/virtualenv/20.13.0/libexec/lib/python3.10/site-packages/virtualenv/__init__.py

But while installing I get this error:

Collecting numpy==1.21.4
  Using cached numpy-1.21.4-cp310-cp310-macosx_11_0_arm64.whl (12.4 MB)
ERROR: Could not find a version that satisfies the requirement opencv-contrib-python-headless==4.1.2.30 (from versions: 3.4.0.14, 3.4.11.45, 3.4.13.47, 3.4.14.51, 3.4.15.55, 3.4.16.57, 3.4.16.59, 3.4.17.61, 4.4.0.46, 4.5.1.48, 4.5.2.52, 4.5.3.56, 4.5.4.58, 4.5.4.60, 4.5.5.62)
ERROR: No matching distribution found for opencv-contrib-python-headless==4.1.2.30

Scripts seems to use requirements_3.6.txt instead of requirements_3.9.txt, while it is recognizing Python 3.10.1 instead of the 3.9.7 version that was set with pyenv. Weird.

Using requirements.txt for Python 3.10.1

Any hints?

[gimmel]

I corrected the version numbers (used "yolk -V appname" to get versions, install with "pip install yolk3k") in requirements.txt but failed with tensorflow. yolk shows version 2.7.0 but "pip install tensorflow" and "pip3 install tensorflow" both fail.

ERROR: Could not find a version that satisfies the requirement tensorflow==2.7.0 (from versions: none)
ERROR: No matching distribution found for tensorflow==2.7.0

[chychkan]

Hi @gimmel! Yeah, we don't have a requirements.txt file for Python 3.10 yet. But also, the M1 chip creates additional issues because it is ARM64 and some of the python packages are not compatible with it. Working on it.

[chychkan]

With https://github.com/chychkan/DeepFaceLab_MacOS/pull/71 we've added partial support for Apple M1 laptops.

[chychkan]

The only remaining issue is with XSeg editor and for that we've opened https://github.com/chychkan/DeepFaceLab_MacOS/issues/77. So, closing this one now.