Importing from sub directory

[cvanelteren]

Describe the bug I have a made a nim package that I wish to import in python to do some plotting. I get an error when I have plotting functions that import the nim package in a subdirectory of the folder. To Reproduce For example, consider the following directory structure

.
├── __pycache__
│   ├── foobar.cpython-311-x86_64-linux-gnu.so
│   └── foobar.hash
├── foobar
│   ├── __pycache__
│   │   ├── foobar.cpython-311-x86_64-linux-gnu.so
│   │   └── foobar.hash
│   ├── foobar.nim
│   ├── foobar.nim.cfg
│   └── foobar.nimble
└── subfolder
    └── subfolder.py

where subofolder/subfolder.py uses nimporter to import foobar. When subfolder.py is put inside the parent directory the code runs fine, but it doesn't when the python file is "below" the package path. Similarly, if the file is anywhere else in the path (e.g. root folder), it runs fine too.

Expected behavior Run code from anywhere in the path

Software Versions

• Python: 3.11
• Nim: 1.6.14
• C Compiler gcc (GCC) 13.1.1 20230714
• Operating System Linux 6.4.4-arch1-1
• CPU Architecture x64

Nimporter Debug Output

cd subfolder && python subfolder.py

Traceback (most recent call last):
  File "/home/casper/projects/test/subfolder/subfolder.py", line 1, in <module>
    import nimporter, foobar
ModuleNotFoundError: No module named 'foobar'

Define NIMPORTER_INSTRUMENT in the environment and post the output below: ?

...

[Pebaz]

Hi @cvanelteren, thanks for submitting a bug report!

This is expected behavior for how imports work in Nimporter. Extension Libraries (folders containing .nimble, .nim.cfg, .nim) are treated as a single importable unit and are akin to a single Python module. As such, to reproduce this in Python, you can remove everything in the foobar/ directory and put an empty __init__.py in it. If you run the subfolder.py file in the same ways that you described, you will get the same behavior because Python won't be able to see the foobar package.

When integrating a Nim extension into a Python project, there are only 2 supported layouts: Extension Modules and Extension Libraries. Here's the layout for Extension Libraries.

I would recommend splitting out your Nim extension into it's own standalone Python package with Nimporter and then you can use it like any other Python library. However, this is not required as you can follow the 2 extension layouts as defined in the README.

I'll close this for now as I believe this should fix it but let me know if this doesn't work for you. 😄

[cvanelteren]

Hi @Pebaz,

Thank you for the extensive answer! Could you elaborate a bit more. In a python module I could install the package and then run the file under a subfolder that is part of the root. That is, the python code would be under (for example) ~./src and some other examples files are under ~./examples, the example files could be run by importing the installed version of the python package/library. This was my intended use here. Hopefully this clears it up a bit more.