TorchStudio / torchstudio

IDE for PyTorch and its ecosystem
https://torchstudio.ai
MIT License
378 stars 27 forks source link

Solving environment: ...working... #2

Closed drkp4 closed 2 years ago

drkp4 commented 2 years ago

Hi - Thank you for this amazing IDEa! Glad we found it. Having trouble setting up environment. Help is appreciated. Happy to provide more info.

Process Downloaded TorchStudio_0.92-Mac_Installer.pkg Installed Successfully Opened TorchStudio TorchStudio Python Environment popup Clicked Install (recommended) Stuck in Solving environment: ...working...

System Information macOS Monterey Version 12.2.1 MacBook Pro (13-inch, M1, 2020) Chip Apple M1 Memory 16 GB

TorchStudio Python Environment Installation Downloading, installing and setting up a Python environment for TorchStudio. This may take up to 5 minutes depending on your download speed, and up to 6GB.

Downloading Python installer (Miniconda3)... Download complete: /Users/drkp4/TorchStudio/python-miniconda3.sh

Installing Python in /Users/drkp4/TorchStudio/python... Installation complete.

Downloading and installing PyTorch and additional packages: nomkl pytorch torchvision torchaudio datasets scipy pandas matplotlib-base python-graphviz paramiko

Collecting package metadata (current_repodata.json): ...working... done Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve. Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): ...working... done Solving environment: ...working...

divideconcept commented 2 years ago

It's not necessarily stuck. On some computers/systems, that "Solving environment" step might take some time. Let me know if it's still stuck after waiting 10 minutes. If it's still stuck, you can install the requirements manually: In macOS open a terminal (Launchpad > Other > Terminal), and type the following line: ~/TorchStudio/python/condabin/conda install nomkl pytorch torchvision torchaudio datasets scipy pandas matplotlib-base python-graphviz paramiko -c pytorch -c nvidia -c huggingface -c conda-forge -c anaconda

drkp4 commented 2 years ago

Thank you! It does seem stuck (more than an hour of waiting)

Tried the following. Should I try a re-install? Thanks so much!

Attempting to Run Conda from Finder drkp4@drkp4s-MacBook-Pro ~ % /Users/drkp4/TorchStudio/python/condabin/conda ; exit; zsh: /Users/drkp4/TorchStudio/python/condabin/conda: bad interpreter: /Users/drkp4/TorchStudio/python.tmp/bin/python: no such file or directory

Saving session... ...copying shared history... ...saving history...truncating history files... ...completed.

[Process completed]

Terminal Suggested Solution drkp4@drkp4s-MacBook-Pro condabin % ~/TorchStudio/python/condabin/conda install nomkl pytorch torchvision torchaudio datasets scipy pandas matplotlib-base python-graphviz paramiko -c pytorch -c nvidia -c huggingface -c conda-forge -c anaconda zsh: /Users/drkp4/TorchStudio/python/condabin/conda: bad interpreter: /Users/drkp4/TorchStudio/python.tmp/bin/python: no such file or directory drkp4@drkp4s-MacBook-Pro condabin %

Terminal Exploration drkp4@drkp4s-MacBook-Pro python % ls LICENSE.txt condabin include python.app ssl bin envs lib share conda-meta etc pkgs shell drkp4@drkp4s-MacBook-Pro python % cd condabin drkp4@drkp4s-MacBook-Pro condabin % ls conda drkp4@drkp4s-MacBook-Pro condabin % cd conda cd: not a directory: conda drkp4@drkp4s-MacBook-Pro condabin % ls conda drkp4@drkp4s-MacBook-Pro condabin % conda install nomkl pytorch torchvision torchaudio datasets scipy pandas matplotlib-base python-graphviz paramiko -c pytorch -c nvidia -c huggingface -c conda-forge -c anaconda zsh: command not found: conda drkp4@drkp4s-MacBook-Pro condabin % conda --help zsh: command not found: conda

sodoherty-ai commented 2 years ago

Same issue:

Model Name: MacBook Pro Model Identifier: MacBookPro18,2 Chip: Apple M1 Max Total Number of Cores: 10 (8 performance and 2 efficiency) Memory: 64 GB

Suggested command line option fails:

/Users/xxx/TorchStudio/python/condabin/conda: bad interpreter: /Users/xxx/TorchStudio/python.tmp/bin/python: no such file or directory

sodoherty-ai commented 2 years ago

For the M1 I had recompile pytorch on the machine using this (in pycharm pro).

conda install -c conda-forge pytorch

When I try to use that environment Torch Studio throws the error:

pytorch 1.10 required. 1.09 installed. 
divideconcept commented 2 years ago

I can repro the issue on my M1 laptop. Investigating...

drkp4 commented 2 years ago

would love to help - happy to connect on discord/slack and troubleshoot live

divideconcept commented 2 years ago

To give you some insights, there was a very recent update (a week ago ?) of the MKL package whose PyTorch depends on. Unfortunately, that latest MKL update is not compatible with M1 machines at all. As a workaround, I added the "nomkl" package during TorchStudio initialisation, which somehow block the use of MKL (but doesn't stop PyTorch from working). Unfortunately, it now seems that combining nomkl and pytorch during install is blocking conda during initialisation... I'm now trying different combinations to see what works/what doesn't.

divideconcept commented 2 years ago

related: https://stackoverflow.com/questions/70830755/intel-mkl-fatal-error-this-system-does-not-meet-the-minimum-requirements-for-us/71190634

drkp4 commented 2 years ago

normal Mac situation! haha.

will try on linux and revert

arnavdas88 commented 2 years ago

I faced the same issue with my i5 Mac. @drkp4 This is what solved my issue :

1.) First, conda needs python.tmp directory for some reason instead of python so I created a symbolic link.

$ cd ~/TorchStudio/
$ ln -s ./python ./python.tmp

Now that ./python.tmp exists, we can use conda --help and other operations...

2.) Add the channel "conda-forge" and update.

$ cd ~/TorchStudio/python/condabin/

$ ./conda config --add channels conda-forge
$ ./conda update conda

3.) Once that is done, I was able to successfully run the command provided by @divideconcept

$ ~/TorchStudio/python/condabin/conda install nomkl pytorch torchvision torchaudio datasets scipy pandas matplotlib-base python-graphviz paramiko -c pytorch -c nvidia -c huggingface -c conda-forge -c anaconda

Although it did give me a small warning in the end, I was able to successfully run TorchStudio after that The WARNING :

| WARNING conda.core.envs_manager:register_env(50): Unable to register environment. Path not writable or missing.
  environment location: /Users/andrew/TorchStudio/python.tmp
  registry file: /Users/andrew/.conda/environments.txt
divideconcept commented 2 years ago

Ok, after testing A LOT of different combinations, I found that the most reliable for mac was to specify explicitly the PyTorch version to install, in this case the version right before the latest update (PyTorch 1.10.1 instead of 1.10.2, because 1.10.2 introduce a dependency on a newer MKL incompatible with M1 macs).

Now building new installers, TorchStudio 0.9.3 should be available later today.

divideconcept commented 2 years ago

The new release is available ! TorchStudio 0.9.3 -> https://torchstudio.ai/download/ Please delete completely your ~/TorchStudio folder before installing and running this one. rm -f -r ~/TorchStudio

sodoherty-ai commented 2 years ago

Just confirming. It installs perfectly now.

drkp4 commented 2 years ago

winner winner chicken dinner! hope to find ways to help the project! thank you so much :)