search

Grid2op / lightsim2grid

LightSim2grid implements a c++ backend targeting the Grid2Op (https://github.com/rte-france/grid2op) platform.
https://lightsim2grid.readthedocs.io/en/latest/
Mozilla Public License 2.0
51 stars 10 forks source link

ImportError: dlopen symbol not found in flat namespace (_klu_analyze) #55

Closed AvisP closed 8 months ago

AvisP commented 1 year ago

Environment

Installation finished without any issues. However when doing

from lightsim2grid.LightSimBackend import LightSimBackend

I am getting the following error message


Cell In [1], line 1
----> 1 from lightsim2grid.LightSimBackend import LightSimBackend

     11 __all__ = ["newtonpf", "SolverType", "ErrorType", "solver"]
     13 # import directly from c++ module
---> 14 from lightsim2grid.solver import SolverType
     15 from lightsim2grid.solver import ErrorType
     17 try:

      1 # Copyright (c) 2020, RTE (https://www.rte-france.com)
      2 # See AUTHORS.txt
      3 # This Source Code Form is subject to the terms of the Mozilla Public License, version 2.0.
   (...)
      8 
      9 # TODO add the other solver now !
     10 __all__ = ["SolverType",
     11            "ErrorType",
     12            "AnySolver",  
   (...)
     16            "SparseLUSolverSingleSlack",
     17            "DCSolver"]

---> 19 from lightsim2grid_cpp import SolverType
     20 from lightsim2grid_cpp import ErrorType
     21 from lightsim2grid_cpp import AnySolver

ImportError: dlopen(......./venvs/L2PRN/lib/python3.10/site-packages/lightsim2grid_cpp.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace (_klu_analyze)```

What might be the issue?
BDonnot commented 1 year ago

Hello,

Have you installed it from source ? If so, from the master branch ? Or from one of the official version (a tag like v0.7.1.post1) ?

It looks like there might be a problem with the compilation of SuiteSparse.

Have you tried to install it from pypi with pip install lightsim2grid (or pip3 install lightsim2grid or something similar - pip command line change name...)

AvisP commented 1 year ago

Hi,

I did only a pip install lightsim2grid while installing. The version is 0.7.1.post1 as this is what I see

Requirement already satisfied: lightsim2grid in ...../L2PRN/lib/python3.10/site-packages (0.7.0.post1)

It didn't mention anything about SuiteSparse during installing....

BDonnot commented 1 year ago

Hummm ok this is super weird.

I do not own a macos machine so debugging it will be hard. I will see what I can do with the script that provide the package.

On the mean time (if you are in a hurry), there are two short-term solutions :

I will try to have a look tomorrow and see what I can do.

AvisP commented 1 year ago

Thanks for your prompt responses. I am using grid2ops and switched to pandapower for the time being but would like to use lightsim2grid later on for performance. I have to use a mac machine unfortunately and don't have access to windows one yet.

I will try to compile it from source by following the instructions. Please let me know if I can be of assistance in the macos debugging. Thanks

BDonnot commented 1 year ago

Yes indeed using pandapower for the tests sounds a good plan.

I will have a look at the scripts thag generate the files installed by pip. And maybe I'll you if you can install other file to check which works if you agree of course.

Thanks

AvisP commented 1 year ago

Hi, Just to update you on this. I was able to install it from the source package and in a new virtual environment by following the custom install option of SparsSuite for macos. However there were some small issues that I resolved locally

After that the following small test on using LightSimBackend worked without any issues

from lightsim2grid.LightSimBackend import LightSimBackend
from lightsim2grid_cpp import SolverType
from lightsim2grid.solver import KLUSolver
from grid2op.gym_compat import GymEnv
import grid2op

env_name = "l2rpn_case14_sandbox"
training_env = grid2op.make(env_name, test=True, backend=LightSimBackend())
gym_env = GymEnv(training_env)
BDonnot commented 1 year ago

Hello,

Thanks for thiq detailed procedure.

I unfortunately did not had time to have a look at this issue (nor any issue regarding lightsim2grid).

I really appreciate you spend some time to find these bugs a'd to report them.

I will update the installation on macos and see if it works after that. (I'll do it in 2023 most likely). And likewise I'll fix the python side with deprecated numpy bool and str

Thanks a lot

Benjamin

BDonnot commented 1 year ago

Hello,

I pushed a new version and test that it works (ie that the package once compiled can be installed and that it runs properly) can you upgrade lightsim2grid to version 0.7.1 and grid2op to version 1.8.1 and let me know if you still have this issue ?

Thanks

(there is a chance the issue is still present as I could not replicate it on any of my machines)

AvisP commented 1 year ago

Hi Benjamin,

I did a test with lighsim2grid 0.7.1 and grid2op 1.8.1. I was able to successfully create a training environment with "l2rpn_case14_sandbox" case without any issues. A pip installation of lightsim2grid fails but I followed the manual instructions and that worked properly. Thanks!

BDonnot commented 1 year ago

Hummm weird.

Can you tell me were you got the information about the compilation of SuiteSparse ? Is there something else than "make" to do that I did not understood ?

Benjamin

AvisP commented 1 year ago

I just followed the steps on the Usage section of lightsim2grid github link. The commands I used were 

git submodule init
git submodule update
make
BDonnot commented 1 year ago

This is weird because it's exactly what is done to compile the "wheel" released on pypi :thinking:

I'll try to have another look

BDonnot commented 1 year ago

Hello @AvisP can you check the solution I explain here:

https://github.com/BDonnot/lightsim2grid/issues/59#issuecomment-1688261097

and tell me if it solves your issue, by any chance ?

AvisP commented 1 year ago

Hi @BDonnot I created a new virtual environment with python 3.10.7 and did what was suggested with pip and got the following error message

WARNING: Requirement 'LightSim2Grid-0.7.3-cp310-cp310-macosx_11_0_arm64.whl' looks like a filename, but the file does not exist
Processing ./LightSim2Grid-0.7.3-cp310-cp310-macosx_11_0_arm64.whl
ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/Users/........./LightSim2Grid-0.7.3-cp310-cp310-macosx_11_0_arm64.whl'
BDonnot commented 1 year ago

Can you try with simply pip install lightsim2grid?

I made the release yesterday on pypi

AvisP commented 1 year ago

So I tried with that command and installed without any errors. But then I started python and did from lightsim2grid.lightSimBackend import LightSimBackend and got the following

from lightsim2grid.lightSimBackend import LightSimBackend
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/paula/Desktop/Projects/venvs/test_lightsim2grid/lib/python3.10/site-packages/lightsim2grid/__init__.py", line 17, in <module>
    from lightsim2grid.solver import SolverType
  File "/Users/paula/Desktop/Projects/venvs/test_lightsim2grid/lib/python3.10/site-packages/lightsim2grid/solver/__init__.py", line 18, in <module>
    from lightsim2grid_cpp import SolverType
ImportError: dlopen(/Users/paula/Desktop/Projects/venvs/test_lightsim2grid/lib/python3.10/site-packages/lightsim2grid_cpp.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '_klu_analyze'
BDonnot commented 1 year ago

Apprently, from here https://github.com/wanadev/mozjpeg-lossless-optimization/issues/5#issuecomment-1658635511

Can you try to upgrade your python version ?

Thanks (it takes sooo long to fix this because I don't own anything using MacOS...)