installing the package via pip fails in python 3.6.[0-4] environment

[antoinerg]

From the root of the project on master, the following fails:

$ docker run -it -v $(pwd):$(pwd) -w $(pwd) python:3.6.4 pip install -e . --quiet
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/local/lib/python3.6/site-packages/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/usr/local/lib/python3.6/site-packages/pip/wheel.py", line 247, in move_wheel_files
    prefix=prefix,
  File "/usr/local/lib/python3.6/site-packages/pip/locations.py", line 141, in distutils_scheme
    d.parse_config_files()
  File "/usr/local/lib/python3.6/site-packages/setuptools/dist.py", line 494, in parse_config_files
    ignore_option_errors=ignore_option_errors)
  File "/usr/local/lib/python3.6/site-packages/setuptools/config.py", line 106, in parse_configuration
    meta.parse()
  File "/usr/local/lib/python3.6/site-packages/setuptools/config.py", line 382, in parse
    section_parser_method(section_options)
  File "/usr/local/lib/python3.6/site-packages/setuptools/config.py", line 355, in parse_section
    self[name] = value
  File "/usr/local/lib/python3.6/site-packages/setuptools/config.py", line 173, in __setitem__
    value = parser(value)
  File "/usr/local/lib/python3.6/site-packages/setuptools/config.py", line 430, in _parse_version
    version = self._parse_attr(value)
  File "/usr/local/lib/python3.6/site-packages/setuptools/config.py", line 305, in _parse_attr
    module = import_module(module_name)
  File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/antoine/plotly/dash-enterprise-auth/dash_enterprise_auth/__init__.py", line 14, in <module>
    import dash_core_components as _dcc
ModuleNotFoundError: No module named 'dash_core_components'

whereas it succeeds with python 3.6.5:

$ docker run -it -v $(pwd):$(pwd) -w $(pwd) python:3.6.5 pip install -e . --quiet
You are using pip version 10.0.1, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

It might be related to https://github.com/plotly/dash-enterprise-auth/issues/5

[antoinerg]

cc @josegonzalez @alexcjohnson

[josegonzalez]

Can we wrap that in a try/ImportError?

Or can we move the logic out of the __init__.py? I'd say that its pretty bad practice to put code in there (although its probably easier for imports).

[antoinerg]

Can we wrap that in a try/ImportError?

What are you suggesting we wrap?

Or can we move the logic out of the init.py? I'd say that its pretty bad practice to put code in there (although its probably easier for imports).

It's worth a try. I'm a little bit confused by the code in here. Is it me or the file _api_requests is not imported/used anywhere?

[alexcjohnson]

we can (and should) move all that stuff out of __init__.py, but that won't fix the issue because __init__.py will just import it from wherever that code goes. And in principle we could wrap the import dash_core_components as _dcc in try/ImportError but if dcc isn't there we're going to have problems later. dcc should come in with dash, which is listed in setup.cfg - does Py 3.6.0 fail to correctly read setup.cfg or something? Or is it just not installed in the right order, and by the end of the installation we have it?

[antoinerg]

does Py 3.6.0 fail to correctly read setup.cfg or something

The use of setup.cfg was introduced in setuptools 30.3.0

Py 3.6.0 has an older version of setuptools so it should choke

$ docker run -it -v $(pwd):$(pwd) -w $(pwd) python:3.6.0 python -c "import setuptools; print(setuptools.__version__)"
28.8.0

Py 3.6.4 is fine:

$ docker run -it -v $(pwd):$(pwd) -w $(pwd) python:3.6.4 python -c "import setuptools; print(setuptools.__version__)"
39.0.1

but it still fail

Possibly: https://bugs.python.org/issue28279

[josegonzalez]

I also mean that the __init__.py should also not do that import either, and the code to use enterprise auth would be:

# auth is dash_enterprise_auth/auth.py
from dash_enterprise_auth import auth

# the methods are now in the auth module
auth.create_logout_button()

This wouldn't be bc-compatible of course.

---

Could we avoid using a setup.cfg? That might also fix the issue. I've never used a cfg file (only setup.py) so not sure of any disadvantages of dropping it.

[antoinerg]

Could we avoid using a setup.cfg? That might also fix the issue. I've never used a cfg file (only setup.py) so not sure of any disadvantages of dropping it.

That's what @alexcjohnson recommended doing and I'm hopeful it will solve the issue. It's not clear setup.cfg can be properly parsed in Python 3.6 as per https://bugs.python.org/issue28279 !

[alexcjohnson]

I also mean that the __init__.py should also not do that import either

API-breaking aside, honestly that seems backward to me - it both makes the package more verbose to use and defers this problem from install time to run time, which would make debugging even harder. Yes I'd prefer we move the functionality to dash_enterprise_auth/auth.py but then I still think __init__.py should import and re-export the public API

[josegonzalez]

I'm personally of the opinion that packages should have no logic in their __init__.py files - this has caused me more issues in production than I care to remember. Numpy/scipy are another package that does this - I recall that scipy depends on numpy during the install but doesn't need to, and neither project wants to fix the underlying issue - and it's quite painful to see in prod.

We don't actually depend on dash but on dcc in this project, though the install_requires points at dash. If there was no logic in that __init__.py and we had the correct import, as long as the package installed, then there wouldn't be a problem.

[alexcjohnson]

The prod deployment argument makes sense, but there must be a way around that without compromising usability for local users. Makes me wonder why the package needs to be imported at all during installation... but this is getting far off topic. Glad it was just a .py vs .cfg issue, it's really useful to know for future reference that we just shouldn't use setup.cfg.