Failure parsing dependences of pypi:packer.py

[bennati]

How to reproduce: run python-inspector --requirement ./requirements.txt --python-version 310 --operating-system linux --json-pdt ./o.json --analyze-setup-py-insecurely where requirements.txt contains packer.py.

This will produce either error

• Exception: Unable to collect setup.py dependencies securely if argument analyze_setup_py_insecurely is False (see this line), or
• error: option --requirement not recognized if argument analyze_setup_py_insecurely is True

These issues are caused by the dependency packer.py having no dependencies of its own: from setup.py

"""
Copyright 2018 Matthew Aynalem

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
from distutils.core import setup
from setuptools import find_packages

setup(
    name='packer.py',
    version='0.3.0',
    author='Matthew Aynalem',
    author_email='maynalem@gmail.com',
    packages=['packerpy'],
    url='https://github.com/mayn/packer.py',
    license='Apache License 2.0',
    description='packer.py - python library to run hashicorp packer CLI commands',
    keywords="hashicorp packer",
    long_description=open('README.rst').read(),
    install_requires=[
    ],
    classifiers=[
        'License :: OSI Approved :: Apache Software License',
        'Programming Language :: Python :: 2',
        'Programming Language :: Python :: 2.7',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.4',
        'Programming Language :: Python :: 3.5',
        'Programming Language :: Python :: 3.6',
    ],
)

[bennati]

I investigated the issue further, in case analyze_setup_py_insecurely is enabled.

The problem happens during the evaluation of packer.py's setup.py:

The setup.py does not contain values for script_name and script_args, so these values are populated by default from argv as script_name = "resolve_cli.py" and script_args = ["--python-version", "310", "--operating-system", "linux", "--json-pdt", "./o.json", "--analyze-setup-py-insecurely"]. This causes the crash because the provided arguments do not match the recognized arguments, which are defined in the Distribution class.

I tried to specify manually these parameters in the setup.py file but i was not able to get it work, does anyone have an idea what to try next?

[pombredanne]

@bennati Thanks... I wonder why this fail "in securely" mode.... and if we should try securely first in this case. Here we parse it perfectly well:

>>> from _packagedcode.pypi import *
>>> p=".cache/python_inspector/extracted_sdists/packer.py-0.3.0/packer.py-0.3.0/setup.py"
>>> from pprint import pprint
>>> pprint(get_setup_py_args(p))
{'author': 'Matthew Aynalem',
 'author_email': 'maynalem@gmail.com',
 'classifiers': ['License :: OSI Approved :: Apache Software License',
                 'Programming Language :: Python :: 2',
                 'Programming Language :: Python :: 2.7',
                 'Programming Language :: Python :: 3',
                 'Programming Language :: Python :: 3.4',
                 'Programming Language :: Python :: 3.5',
                 'Programming Language :: Python :: 3.6'],
 'description': 'packer.py - python library to run hashicorp packer CLI '
                'commands',
 'keywords': ['hashicorp', 'packer'],
 'license': 'Apache License 2.0',
 'name': 'packer.py',
 'url': 'https://github.com/mayn/packer.py',
 'version': '0.3.0'}

But there are no deps (no values listed in install_requires) and this is likely why this fails.

The other thing is for the insecure mode, that this uses the older distutils and we mock setuptools! If I patch this way:

diff --git a/src/python_inspector/setup_py_live_eval.py b/src/python_inspector/setup_py_live_eval.py
index e9515e5..cca3241 100755
--- a/src/python_inspector/setup_py_live_eval.py
+++ b/src/python_inspector/setup_py_live_eval.py
@@ -12,6 +12,7 @@
 import os
 import re
 import sys
+import distutils

 try:
     import configparser
@@ -54,7 +55,7 @@
     setup_requires = {}
     # change directory to setup.py path
     with pushd(os.path.dirname(setup_file)):
-        with mock.patch.object(setuptools, "setup") as mock_setup:
+        with mock.patch.object(distutils.core, "setup") as mock_setup:
             sys.path.append(os.path.dirname(setup_file))
             g = {"__file__": setup_file, "__name__": "__main__"}
             with open(setup_file) as sf:

then $ python-inspector --requirement r.txt --python-version 310 --operating-system linux --json-pdt ./o.json --analyze-setup-py-insecurely works perfectly well

So we would need to patch either setuptools or distutils.core... this could be either based on a try/except or testing first if the file imports setuptools or distutils

[bennati]

Thanks, that fixes the issue, pushed to https://github.com/nexB/python-inspector/pull/120