Open rahimikia opened 2 years ago
This is probably due to pip dependency resolver. try upgrading pip to latest pip install --upgrade pip
and run it again.
If that fails, you could try installing packages one by one cat requirements.txt | xargs -n 1 pip install
(you may end up with some incompatibilities though)
This is the common issue, to limit the time of installation you can set some constraints (>=, ~=) to some of the packages that the installation is blocking, you can check the previous resolver pip install --use-feature=2020-resolver -r requirements.txt
Also there are some hints in here https://pip.pypa.io/en/stable/topics/dependency-resolution/#possible-ways-to-reduce-backtracking
I had a similar problem earlier as well. What worked for me was that after cloning the repo
I ran pip install git+https://github.com/EleutherAI/lm-evaluation-harness/
then i went to the requirements.txt, removed git+https://github.com/EleutherAI/lm-evaluation-harness/
, uvicorn
and fastapi
After that I ran pip install -r requirements.txt
and everything got installed.
Running 'requirements.txt' for fine-tuning on TPU takes hours for me without any end. It's always looking at multiple versions of different packages (like multidict), and each one takes hours to finish. Any solution for this?