Nimporter fails to import due to undefined symbol "__builtin_ssubll_overflow" when using GCC on CentOS 7.5

[AlexisGoodfellow]

So, I got this really ugly stack trace when trying to import the basic nim_math.nim example code into my python project. The relevant sections of the project tree look like:

root/
    nim/
        nim_math.nim
        __pycache__/
            nim_math.nim.hash
            nim_math.so
    src/
        app.py

And the relevant contents of app.py are:

...
import nimporter
import nim.nim_math
...

Here's some more data about the environment that isn't available in the stack trace: I'm using a CentOS 7.5 base docker image with GCC version 4.8.5. I've also replicated this same failure with the same docker image on GCC versions 7.x and 8.x.

/pyenv/versions/3.8.0/envs/my-new-project/lib/python3.8/site-packages/nimporter.py:1117: in find_spec
    return Nimporter.import_nim_code(fullname, path, library=False)
/pyenv/versions/3.8.0/envs/my-new-project/lib/python3.8/site-packages/nimporter.py:819: in import_nim_code
    cls.__validate_spec(spec)
/pyenv/versions/3.8.0/envs/my-new-project/lib/python3.8/site-packages/nimporter.py:887: in __validate_spec
    raise NimporterException(error_message) from import_error
E   nimporter.NimporterException: Error importing /app/nim/__pycache__/nim_math.so
E   Error Message:
E
E       /app/nim/__pycache__/nim_math.so: undefined symbol: __builtin_ssubll_overflow
E
E   Python Version:
E
E       3.8.0 (default, Dec 16 2019, 22:06:06) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
E
E   Nim Version:
E
E       Nim Compiler Version 1.2.0 [Linux: amd64]
E
E   CC Version:
E
E       <Error getting version>
E
E   Installed CCs:
E
E       {'gcc': PosixPath('/usr/bin/gcc')}
E
E   Please help improve Nimporter by opening a bug report at: https://github.com/Pebaz/nimporter/issues/new and submit the above information along with your description of the issue.

I'm not sure if the issue is with nimporter, nimpy, the nim compiler, or gcc itself. It looks like this is an issue with the compiled output, so nimporter might be off the hook here. If that's the case, I'll have to file this same issue one layer deeper down the stack - which I guess would involve pestering the folks who develop nimpy.

I'm hoping this is an easy fix though - maybe there's some compiler switch I'm missing that could make this all go away. Thanks for the help (and the awesome project!)

[cgarciae]

Hey Alexis, are you using Nimble? For this to work I think you need a *.nimble file under root/.

[AlexisGoodfellow]

I was not packaging my nim code with nimble. I can try adding a .nimble file, but my understanding from the docs is that for one-off scripts like the nim_math.nim example it wouldn't be necessary.

But I guess it can't hurt - I'll give it a shot!

[AlexisGoodfellow]

I can confirm that this still happens even when there is a *.nimble file matching the project name (both directly under root, and also under nim). I've tried all the permutations of directory/file names, and none of them have made this error go away

So much for that :(

[AlexisGoodfellow]

Given the information I have in front of me, it would appear that there's something going awry at the compiler layer. Pretty unlikely it's a problem with GCC, so maybe there's an issue with the transpilation from Nim to C?

[cgarciae]

@AlexisGoodfellow can you share your Nimble file?

[Pebaz]

Nimporter author checking in here.

I will look over the information present in this issue and give my recommendation soon.

Thank you so much for filing a bug report! 🌴

[Pebaz]

I was able to copy your folder structure and run successfully on:

• CentOS 8.1.1911 (Core)
• GCC 8.3.1
• Python 3.6.8
• Nimporter 1.0.1
• Nim 1.2.0
• Everything was 64 bit

How are you executing the app.py? I used:

$ python3 -m src.app

Without the above command, app.py could not see the nim folder in order to import nim_math.

Although Nimporter was designed to be used flexibly, it seems as if your source code and Nim extensions are separated (nim and src).

The original intention is to place your Nim code alongside the Python code.

If you need to use dependencies for your Nim code, just put the Nim file in a folder of the same name alongside your other Python code and then add a <foldername>.nimble file with the relevant dependency info.

Again, Nimporter usage was designed to be flexible but just something I noticed.

If you can provide me the relevant info for running in the same Docker container as you are using, I would be happy to debug why it is not working.

My best guess at this point is the outdated GCC version 4.8.5. I have not tested Nimporter with this version and it may be the problem.

Some more insight into what is causing the issue is that Nim compiles the Nim source to C and then successfully compiled/linked this to a .so. However, when Python tried to import it, it failed with the above error message.

To further test this, you can cd into the __pycache__ dir containing the .so file and then open a Python prompt. Then just directly type: import nimporter, nim_math and the error should repeat itself. If not, I'd be interested to see what the new error is.

I will await your response!

[AlexisGoodfellow]

I'm still looking into this! This has taken a while to suss out because our docker image build process is much more complex than I anticipated.

My current guess is that the core problem is that python itself was compiled with gcc 4.8.5, because this still happens when I use scl to update the version of gcc in the docker container to version 9.1.1.

I've had to put this to the side for a while due to other obligations, but I will investigate this more later in the week.