No callable attribute: exec [ValueError]

pietroppeter commented 2 years ago

with a recent installation of nimpy on Mac (M1), the following does not work:

import nimpy

let py = pyBuiltinsModule()
discard py.exec("sum([1, 2, 3])")

and throws error

.nimble/pkgs/nimpy-0.2.0/nimpy.nim(842) callMethodAux
Error: unhandled exception: No callable attribute: exec [ValueError]

the only mention I found of exec in nimpy repo is in this test: https://github.com/yglukhov/nimpy/blob/1963283148d01f46ac789a83412afbb5d375cfad/tests/tpyfromnim.nim#L251 (but exec is a command that works in python interpreter). I found out about the possibility of using exec from this: https://github.com/pietroppeter/nimib/pull/83

note that the readme example works fine

import nimpy

let os = pyImport("os")
echo "Current dir is: ", os.getcwd().to(string)

# sum(range(1, 5))
let py = pyBuiltinsModule()
let s = py.sum(py.range(0, 5)).to(int)
assert s == 10

other info:

Nimpy version 0.2.0
nim version 1.6.0
python version 3.9.10
checked with find_libpython that the lib python that should be used is the 3.9 one: /usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/Python

yglukhov commented 2 years ago

What does echo pyImport("sys").paths say?

pietroppeter commented 2 years ago

that one:

/Users/pietroppeter/.nimble/pkgs/nimpy-0.2.0/nimpy.nim(867) getAttr
/Users/pietroppeter/.nimble/pkgs/nimpy-0.2.0/nimpy/py_utils.nim(98) raisePythonError
Error: unhandled exception: <type 'exceptions.AttributeError'>: 'module' object has no attribute 'paths' [Exception]

but I guess it might have been path and this goes with Python 2.7!

['/usr/lib/python27.zip', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-darwin', '/usr/lib/python2.7/plat-mac', '/usr/lib/python2.7/plat-mac/lib-scriptpackages', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/Library/Python/2.7/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC']

I guess this is it, how do I fix it? the Mac came with python 2.7 for python but I did change the alias to have it use python3

yglukhov commented 2 years ago

You can specify path to libpython using nimpy.py_lib.pyInitLibPath. Also see #171.

pietroppeter commented 2 years ago

tried both paths below (the first is the one given me by find_libpython, the second it looked reasonable to me by looking around in the folders):

import nimpy
import nimpy / py_lib

#pyInitLibPath "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/Python"
pyInitLibPath "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/libpython3.9.dylib"

let py = pyBuiltinsModule()
discard py.exec("sum([1, 2, 3])")

but the result is always an error of loading like this

Error: unhandled exception: Could not load libpython. Tried /usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/libpython3.9.dylib

not sure what I should use...

yglukhov commented 2 years ago

Not sure, could it be because of architecture mismatch?

file /usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/libpython3.9.dylib
file $MY_EXECUTABLE

pietroppeter commented 1 year ago

I went back this (sorry for the delay) and I think back then I had indeed an architecture mismatch (default python on Mac M1 is x86_64). I installed an arm python and now I have both my new python and my nim executable that are both arm (checked with file see below).

I do get a different error now. For reference now I compile and run:

import nimpy
import nimpy / py_lib

pyInitLibPath "/Library/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib"

let py = pyBuiltinsModule()
discard py.exec("sum([1, 2, 3])")

and the error is:

/Users/pietroppeter/play_nimpy/nimpyissue261.nim(10) nimpyissue261
/Users/pietroppeter/.nimble/pkgs/nimpy-0.2.0/nimpy.nim(845) callMethodAux
/Users/pietroppeter/.nimble/pkgs/nimpy-0.2.0/nimpy/py_utils.nim(98) raisePythonError
Error: unhandled exception: <class 'SystemError'>: frame does not exist [Exception]

checking architecture with file:

❯ file /Library/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib
/Library/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit dynamically linked shared library x86_64] [arm64]
/Library/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib (for architecture x86_64):    Mach-O 64-bit dynamically linked shared library x86_64
/Library/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib (for architecture arm64):     Mach-O 64-bit dynamically linked shared library arm64

❯ file nimpyissue261
nimpyissue261: Mach-O 64-bit executable arm64

dmknght commented 9 months ago

I'm having the same issue here. I used the find_libpython and added pyInitLibPath. My test code:

import nimpy
import nimpy / py_lib

pyInitLibPath "/usr/lib/x86_64-linux-gnu/libpython3.11.so.1.0"
let py = pyBuiltinsModule()
discard py.exec("sum([1, 2, 3])")

Output:

/tmp/test.nim(8)         test
/home/dmknght/.nimble/pkgs/nimpy-0.2.0/nimpy.nim(850) callMethodAux
/home/dmknght/.nimble/pkgs/nimpy-0.2.0/nimpy/py_utils.nim(98) raisePythonError
Error: unhandled exception: <class 'SystemError'>: frame does not exist [Exception]
Error: execution of an external program failed: '/tmp/test '

I don't really think this problem comes from different architecture

Update: To make sure pyBuiltinsModule() worked normally, i tried echo py.bin(3) to test and ofc it worked. I tried digging a bit and I found that code use pybind could have this problem too https://stackoverflow.com/questions/53115558/compile-and-execute-ast-using-pybind11-or-python-c-api

Update 2: According to the stackoverflow link, apparently we have to pass py.globals() into eval or exec. However, it showed the NULL exception (just like in the comments). I didn't find a way to execute code properly. But here's my code

import nimpy

let
  py = pyBuiltinsModule()
  modl_builtin = pyImport("builtins")

echo py.eval("x+1", py.globals())
echo modl_builtin.exec("x + 1", py.globals())

dmknght commented 9 months ago

So here's the working code of mine, Nim 1.6.0, nimpy 0.2.0

import nimpy

let
  py = pyBuiltinsModule()

echo py.eval("2+1", py.dict())

Solution found at https://blogs.mathworks.com/loren/2020/03/03/matlab-speaks-python/

dreamflow commented 3 months ago

got exec() working 🙂

import nimpy
import nimpy/py_lib
from std/envvars import getEnv , putEnv

let pythonDll_dirPath = "C:/python"
let pythonDll_filePath = pythonDll_dirPath & "/python3.dll"

let pathEnvVar_name = "Path"
let pathEnvVar_value = pythonDll_dirPath & ";" & getEnv( pathEnvVar_name )
putEnv( pathEnvVar_name , pathEnvVar_value )
    # necessary for pyInitLibPath() to work

pyInitLibPath( pythonDll_filePath )

let sys_pyModule = pyImport("sys")
let builtins_pyModule = pyImport("builtins")

var topLevelDunderMain_pyModule = sys_pyModule.modules["__main__"]
var dunderDictOfTopLevelDunderMain_pyDict
    = builtins_pyModule.getattr( topLevelDunderMain_pyModule , "__dict__" )

discard builtins_pyModule.exec( "print(42)" , dunderDictOfTopLevelDunderMain_pyDict )

exec() doesn't work if its surrounding globals() is empty , what surprisingly is the case here :

echo( builtins_pyModule.str( pyGlobals() ) )
  -> None

therefore a working globals() dict is passed as the globals param into exec() .

my environment : Win10 pro 22H2 python 3.12.4 nimpy (2024-06-29 installed) Nim 2.1.1 "nightly build on 2024-06-10 for branch devel" gcc (MinGW-W64 x86_64-ucrt-posix-seh, built by Brecht Sanders, r1) 13.3.0

yglukhov / nimpy

No callable attribute: exec [ValueError] #261