Trouble following packaging libraries tutorial

[derekwong9]

Thank you for providing feedback on Python packaging!

To help us help you, please fill out as much of the following as you can. If a question is not relevant, feel free to skip it.

1. What is your operating system and version? Windows 10

2. What is your Python version? Python 3.6.5 |Anaconda custom (64-bit)

3. What version of pip do you have? pip 18.0

4. Could you describe your issue in as much detail as possible?

I have created a setup.py by following the guide and run command python setup.py sdist bdist_wheel then I run pip install package pip install lux-0.0.0-py3-none-any.whl

however upon import either on REPL or jupyter notebook I cannot see any of the modules, I run the following

dir(lux) ['__doc__', '__loader__', '__name__', '__package__', '__path__', '__spec__']

the install is basically not picking up any modules. in my setup.py i am importing find_packages from setuptools import setup, find_packages packages=find_packages()

file structure is as follows

lux/
├── lux/
│   ├── mod_1/
│   │   ├── __init__.py
│   │   ├── mod_1.py
│   │   └── more_code.py
│   ├── mod_2/
│       ├── __init__.py
│       ├── cool_things.py
│       └── more_code2.py
├── build/
├── dist/
├── lux.egg.info/
├── setup.py
├── README.md
└── requirements.txt

I have no idea why it wont find the underlying modules. I have read that init.py can cause issues on python 3.3+ so i have tried both with and without init.py at the mod_x/ levels but both give me the same issue.

[merwok]

Is your goal to release two packages (usable by doing import mod_1 and import mod_2), or one (import lux, from lux.mod_2 import cool_things)?

[derekwong9]

1 package, lux.mod1 lux.mod2

On Mon, Aug 27, 2018, 05:54 Éric Araujo notifications@github.com wrote:

Is your goal to release two packages (usable by doing import mod_1 and import mod_2), or one (import lux, from lux.mod_2 import cool_things)?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/pypa/packaging-problems/issues/193#issuecomment-416075290, or mute the thread https://github.com/notifications/unsubscribe-auth/AF595EYf21XgrbJ8-4_ZEsCKuAIijv4Xks5uUxkwgaJpZM4WKvK1 .

[benoit-pierre]

If your testing from your source directory, the lux module that get picked up is probably the one from the source directory (thanks to implicit namespace packages support in Python 3.3+). With the latest version of setuptools, you can change your setup.py to use find_namespace_packages instead of find_packages. Better yet (compatible with Python 2, plus explicit is better than implicit), just add an empty __init__.py in the lux directory too, and it will get picked up by find_packages.

[benoit-pierre]

Also note that using import lux; dir(lux) will not list mod_1 or mod_2, unless they have been explicitly imported too.

[derekwong9]

I tried adding an empty __init__.py to the lux directory, the 2nd tier lux directory. It did not change anything. I am still unable to access any of the modules under lux. if dir(lux) cannot see any imports other than explicit, is there a better way to see what modules/folders/classes are available?

[benoit-pierre]

What is it that you are trying to do exactly? The standard way to access a module is to import it.

If you really want to list lux sub-modules without importing them, you could try using pkgutil:

>>> import lux
>>> import pkgutil
>>> print('\n'.join(map(repr, pkgutil.walk_packages(lux.__path__))))
ModuleInfo(module_finder=FileFinder('[...]/lux'), name='mod_1', ispkg=True)
ModuleInfo(module_finder=FileFinder('[...]/lux'), name='mod_2', ispkg=True)

[benoit-pierre]

Note that independently of your issue with "listing" the contents of the lux package, you need lux/__init__.py or to use find_namespace_packages if you want lux and its contents to be correctly included in your wheel (and installed...).

[derekwong9]

i did add a lux/__init___.py for the python 2 compatibility as recommended above , using this command print('\n'.join(map(repr, pkgutil.walk_packages(lux.__path__)))) i can see the modules inside with given output

ModuleInfo(module_finder=FileFinder('C:\\Anaconda3\\lib\\site-packages\\lux'), name='blotter', ispkg=True)
ModuleInfo(module_finder=FileFinder('C:\\Anaconda3\\lib\\site-packages\\lux'), name='config', ispkg=True)

so now lets say i am trying to instantiate a class lux.blotter.Blotter via REPL/Notebook I cannot. I get AttributeError: module 'lux' has no attribute 'blotter'

I have gotten a bit further now with this additional __init__.py but now I am not sure how to actually call or instantiate classes / methods from these folders that live within lux.

PS @benoit-pierre and @merwok thank you for the lovely support helping a newbie with his first package.

[benoit-pierre]

Again, to access a module, you need to import it, for example:

import lux.blotter

lux.blotter.Blotter()

[derekwong9]

import lux.blotter

ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-71-3709da34cf10> in <module>()
----> 1 import lux.blotter

C:\Anaconda3\lib\site-packages\lux\blotter\__init__.py in <module>()
----> 1 from blotter.blotter import *
      2 

ModuleNotFoundError: No module named 'blotter'

[merwok]

1. You need to make lux a package → add empty lux/__init__.py (same level as mod_1 / blotter)
2. The find_packages function in setup.py should find lux in the current directory without more config
3. In your subpackages and submodules, you have to use the proper imports: either from lux.blotter import * (I assume that you have blotter.py next to __init__.py in the blotter subpackage) or from .blotter import *

(I don’t personally understand why people put code in thing/thing.py and then import in thing/__init__.py: just put the code in thing/__init__.py ! or if you have independently useful submodules, don’t import in __init__ !)