AST-based root module importmap, static typehints, and CI check

[Helveg]

Is your feature request related to a problem? Please describe. Users are very used to importing everything they need from the main namespace import nest then using nest.X. To enable this nest has always imported all of its submodules, to expose these references in the main module.

In the past this has already caused some headaches with circular references, dynamic attributes, and code execution at import time. The fixes of which increased the complexity of the main module.

Initial tests during the hackathon also showed that there was about a 1s cost associated with import nest.

The NEST codebase also doesn't follow PEP8 conventions of prefixing private modules with an underscore, so we're technically endorsing users to import nest.ll_api, or nest.lib.hl_api_spatial. This means we promise to support them as stable public APIs and can't make any changes to any part of the organization of the NEST interface without technically introducing breaking major changes.

Describe the solution you'd like

1. We prefix all modules with an underscore, indicating them as private, not to be imported from by external users, and list their public members in each submodule's __all__.
2. import nest would use the ast stdlib to retrieve the __all__ of each nest submodule, and creates a map between each exported member, and the module it should be loaded from.
3. The root nest module simplifies to implement just a __getattr__, __setattr__ and __dir__ method.They will fulfill 2 mechanisms:
   • Kernel attributes
   • Lazy loading of the submodules whenever one of their public attributes is accessed (readonly)
4. Add static type hints for each attribute that can be accessed
5. Run a CI check that analyses the __annotations__ of nest, and asserts that the total set of __all__ of the submodules and the kernal attributes all occur in the __annotations__, enforcing that each public attribute gets a typehint.

Additional context Initial tests showed that import nest was reduced to 8e-05 seconds. This is because no code except for __init__.py will actually be accessed on import. On very slow filesystems like WSL this went up to 40ms.

[Helveg]

One point of discussion: We can't get rid of the module object manipulation as long as magical attributes need to be set directly on nest, e.g. nest.resolution = 0.1. If perhaps we group these attributes under another object, e.g. nest.kernel.resolution, or nest.attributes.resolution, then we can remove all of the module magic in favor of PEP562 module level __getattr__ and __dir__.

[github-actions[bot]]

Issue automatically marked stale!