search

ronaldoussoren / modulegraph

modulegraph determines a dependency graph between Python modules primarily by bytecode analysis for import statements. modulegraph uses similar methods to modulefinder from the standard library, but uses a more flexible internal representation, has more extensive knowledge of special cases, and is extensible.
MIT No Attribution
38 stars 4 forks source link

Doesn't find packages when pkgutil.extend_path() is used #42

Open ronaldoussoren opened 7 years ago

ronaldoussoren commented 7 years ago

Original report by Michael Root (Bitbucket: mike1158, GitHub: mike1158).

We use pkgutil.extend_path() frequently to allow multiple module packages in use at our company to all share a common top-level company namespace.

modulegraph.find_modules only considers the first package in the namespace.

ronaldoussoren commented 5 years ago

Original comment by Ronald Oussoren (Bitbucket: ronaldoussoren, GitHub: ronaldoussoren).

Are you using pkgutil.extend_path instead of setuptools style namespace packages or in some other way?

The good news is that modulegraph2 (a rewrite of modulegraph using modern python 3 APIs) does support pkgutil.extend_path.

The bad news is that automatically detecting the use of this API is close to impossible using the modulegraph code base, and I'll therefore probably will never fix this issue here.

ronaldoussoren commented 5 years ago

Original comment by Michael Root (Bitbucket: mike1158, GitHub: mike1158).

Hi, I'd forgotten all about this! We use a common namespace for all our in-house code, but we also have several sub-packages that each get versioned independently. Each of those sub-packages is in a separate path, so we put this in the init.py at the top level of those packages to allow them all to share that common namespace:

from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

Our use case for modulegraph was to try and find seldom-used packages so that we can factor them out, or to identify what might be using an API that is about to undergo some breaking change.

Does that info help at all?