search

pyupio / dparse

A parser for Python dependency files
MIT License
61 stars 23 forks source link

Add support for setup.py #2

Open jayfk opened 7 years ago

jhermann commented 7 years ago

After calling ./setup.py egg_info -e «tmpdir», requires.txt can be fed into a pip requirements parser (though there might be subtle differences).

pombredanne commented 7 years ago

@jhermann the main issue there is that you are executing a setup.py to get there which is fraught with so many perils (not the proper system reqs, python version, etc, etc). See https://github.com/nexB/scancode-toolkit/issues/253#issuecomment-310714287 for some pointers and approaches the main serious ones being either: AST parsing or mock execution

pombredanne commented 7 years ago

@jayfk do you plan to include in this ticket pbr setup.cfg which is popular on openstack and setup.cfg in general too?

pombredanne commented 7 years ago

FWIW, this is an example of setup.py that is hard to make sense: https://github.com/jakubroztocil/httpie/blob/f1d4861faed6486a356175ce9f92d4da96ddba01/setup.py

  1. an AST approach will have a harder time to get the test and install requirements, defined as variables
  2. a mock or AST will have a hard time to get some deps that are Python version- and OS-specific
  3. even a full live execution would be subject to the problem in 2.

I am not sure the complexity is worth handling in such a case, at least for a start.

jayfk commented 7 years ago

@jayfk do you plan to include in this ticket pbr setup.cfg which is popular on openstack and setup.cfg in general too?

This basically looks like a tox.ini: https://github.com/openstack-dev/pbr/blob/master/setup.cfg. Shouldn't be a problem to include it.

an AST approach will have a harder time to get the test and install requirements, defined as variables a mock or AST will have a hard time to get some deps that are Python version- and OS-specific even a full live execution would be subject to the problem in 2.

On setup.py files that come with a wheel we might be able to extract metadata from it. I know that some people are working on this but for the time being using an AST approach is probably the sanest thing we could to.