New pip resolver breaks installation

[litghost]

The new pip resolver breaks the current requirements.txt. In particular, installing a package in edit mode is not compatible with git URL eggs.

[litghost]

For example:

-e third_party/fasm

cannot co-exist with:

fasm@git+git://github.com/SymbiFlow/fasm.git#egg=fasm

[mithro]

Links?

[litghost]

https://blog.python.org/2020/11/pip-20-3-release-new-resolver.html

https://source.cloud.google.com/results/invocations/2bfc97e7-90a0-4fd6-a98e-c9ed41905dbd/log :

failed
Pip subprocess error:
    ERROR: Command errored out with exit status 1:
     command: /tmpfs/src/github/symbiflow-arch-defs-presubmit-ice40/env/conda/envs/symbiflow_arch_def_base/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmpfs/tmp/pip-install-q7oax5kg/cairosvg_fb417a44c8bd431eab92bf1fd53f99af/setup.py'"'"'; __file__='"'"'/tmpfs/tmp/pip-install-q7oax5kg/cairosvg_fb417a44c8bd431eab92bf1fd53f99af/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmpfs/tmp/pip-pip-egg-info-rbidigtk
         cwd: /tmpfs/tmp/pip-install-q7oax5kg/cairosvg_fb417a44c8bd431eab92bf1fd53f99af/
    Complete output (9 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmpfs/tmp/pip-install-q7oax5kg/cairosvg_fb417a44c8bd431eab92bf1fd53f99af/setup.py", line 23, in <module>
        import cairosvg
      File "/tmpfs/tmp/pip-install-q7oax5kg/cairosvg_fb417a44c8bd431eab92bf1fd53f99af/cairosvg/__init__.py", line 27, in <module>
        from . import surface
      File "/tmpfs/tmp/pip-install-q7oax5kg/cairosvg_fb417a44c8bd431eab92bf1fd53f99af/cairosvg/surface/__init__.py", line 23, in <module>
        import cairo
    ModuleNotFoundError: No module named 'cairo'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output

Python Insider: Releasing pip 20.3, featuring new dependency resolver

Develop your code on the Google Cloud Platform.

[mithro]

That looks like an issue with cairo?

[litghost]

It's not. It's an issue with conflicts and the fact that the new pip resolver is insane. It will try to back track packages an infinite number of versions, when it really just needs to stop.

[litghost]

For example:

INFO: pip is looking at multiple versions of cairosvg to determine which version is compatible with other requirements. This could take a while.
Collecting cairosvg
  Downloading CairoSVG-2.4.2-py3-none-any.whl (50 kB)
  Downloading CairoSVG-2.4.1-py3-none-any.whl (50 kB)
  Downloading CairoSVG-2.4.0-py3-none-any.whl (50 kB)
  Downloading CairoSVG-2.3.1-py3-none-any.whl (51 kB)
  Downloading CairoSVG-2.3.0-py3-none-any.whl (51 kB)
  Downloading CairoSVG-2.2.1-py3-none-any.whl (52 kB)
  Downloading CairoSVG-2.2.0-py3-none-any.whl (53 kB)
INFO: pip is looking at multiple versions of fasm to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of edalize to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of cairosvg to determine which version is compatible with other requirements. This could take a while.
  Downloading CairoSVG-2.1.3-py3-none-any.whl (101 kB)
  Downloading CairoSVG-2.1.2-py3-none-any.whl (101 kB)
  Downloading CairoSVG-2.1.1-py3-none-any.whl (101 kB)
  Downloading CairoSVG-2.1.0-py3-none-any.whl (101 kB)
  Downloading CairoSVG-2.0.3.tar.gz (39 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
  Downloading CairoSVG-2.0.2.tar.gz (39 kB)
  Downloading CairoSVG-2.0.1-py3-none-any.whl (101 kB)
  Downloading CairoSVG-2.0.0-py3-none-any.whl (100 kB)
  Downloading CairoSVG-1.0.22.tar.gz (30 kB)
  Downloading CairoSVG-1.0.21.tar.gz (30 kB)
  Downloading CairoSVG-1.0.20.tar.gz (30 kB)
  Downloading CairoSVG-1.0.19.tar.gz (30 kB)
  Downloading CairoSVG-1.0.18.tar.gz (30 kB)
  Downloading CairoSVG-1.0.17.tar.gz (30 kB)
  Downloading CairoSVG-1.0.16.tar.gz (30 kB)
  Downloading CairoSVG-1.0.15.tar.gz (30 kB)
  Downloading CairoSVG-1.0.14.tar.gz (30 kB)
  Downloading CairoSVG-1.0.13.tar.gz (29 kB)
  Downloading CairoSVG-1.0.12.tar.gz (29 kB)
  Downloading CairoSVG-1.0.11.tar.gz (29 kB)
  Downloading CairoSVG-1.0.10.tar.gz (29 kB)
  Downloading CairoSVG-1.0.9.tar.gz (29 kB)
  Downloading CairoSVG-1.0.8.tar.gz (29 kB)
  Downloading CairoSVG-1.0.7.tar.gz (29 kB)
  Downloading CairoSVG-1.0.6.tar.gz (29 kB)
  Downloading CairoSVG-1.0.5.tar.gz (29 kB)
  Downloading CairoSVG-1.0.4.tar.gz (28 kB)
  Downloading CairoSVG-1.0.3.tar.gz (28 kB)
  Downloading CairoSVG-1.0.2.tar.gz (28 kB)
  Downloading CairoSVG-1.0.1.tar.gz (28 kB)
  Downloading CairoSVG-1.0.tar.gz (28 kB)
  Downloading CairoSVG-0.5.tar.gz (24 kB)
  Downloading CairoSVG-0.4.4.tar.gz (22 kB)
  Downloading CairoSVG-0.4.3.tar.gz (22 kB)
  Downloading CairoSVG-0.4.2.tar.gz (22 kB)

[mithro]

Obtaining file:///tmpfs/src/github/symbiflow-arch-defs-presubmit-ice40/third_party/fasm (from -r file:requirements.txt (line 22))

verse

Collecting fasm@ git+git://github.com/SymbiFlow/fasm.git#egg=fasm
  Cloning git://github.com/SymbiFlow/fasm.git to /tmpfs/tmp/pip-install-q7oax5kg/fasm_c8f0f877596f4d21b4289306a247f2f4

[mithro]

I think just adding a fasm@ in front of the -e third_party/fasm might work?

[litghost]

Possible, but also unneeded. Anyways, the problem also appears to be one of the following packages:

-e third_party/python-sdf-timing
-e third_party/python-symbiflow-v2x
-e third_party/vtr-xml-utils

I'm bisecting to see which is the problem.

[litghost]

The problem lies in -e third_party/python-symbiflow-v2x

[mithro]

It does seem like the correct format for a requirement should be <name> or <name>@<location>.

[mithro]

It seems like the @ syntax is actually evil and should not be used. It seems to mean 'at this exact version'?