Python ctypes.CDDL compiled binary loading error

dennis8 commented 1 year ago

I'm trying to get bridgestan to work on Windows 10 Version 10.0.19045 Build 19045. I've downloaded both the Pypi version (through pip install) and the git clone version. I can compile the .stan code to a .so binary and .hpp file with make test_models/bernoulli/bernoulli.stan. However the compiled binary will not load into Python with the class StanModel:

import bridgestan as bs
import numpy as np

stan = "../test_models/bernoulli/bernoulli.stan"
data = "../test_models/bernoulli/bernoulli.data.json"

model = bs.StanModel.from_stan_file(stan, data)

print(f"This model's name is {model.name()}.")
print(f"It has {model.param_num()} parameters.")

x = np.random.random(model.param_unc_num())
q = np.log(x / (1 - x))  # unconstrained scale
lp, grad = model.log_density_gradient(q, jacobian=False)
print(f"log_density and gradient of Bernoulli model: {(lp, grad)}")

Traceback (most recent call last):
  File "...\bridgestan_test\example.py", line 26, in <module>
    model = bs.StanModel.from_stan_file(stan, data)
  File "...\venv\lib\site-packages\bridgestan\model.py", line 244, in from_stan_file
    return cls(
  File "...\venv\lib\site-packages\bridgestan\model.py", line 71, in __init__
    self.stanlib = ctypes.CDLL(self.lib_path)
  File "...\Python310\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module '...\.bridgestan\bridgestan-2.0.0\test_models\bernoulli\bernoulli_model.so' (or one of its dependencies). Try using the full path with constructor syntax.

I have tried to trace where it goes wrong and the following also doesn't seem to work:

from pathlib import Path
BS_FILEPATH = r"C:\Users\MyUsername\.bridgestan\bridgestan-2.0.0\python"
compiled = Path(BS_FILEPATH).parent / "test_models/bernoulli/bernoulli_model.so"
print(compiled.exists()) # Returns True

import ctypes
stanlib = ctypes.CDLL(str(compiled))

FileNotFoundError: Could not find module '....bridgestan\bridgestan-2.0.0\test_models\bernoulli\bernoulli_model.so' (or one of its dependencies). Try using the full path with constructor syntax.

I abbreviated my filepaths, but they seem correct nonetheless.

This results the same error. Any ideas on how to solve this problem?

WardBrian commented 1 year ago

The trouble on Windows is in the next part of the error message (or one of its dependencies). When you load a library on Windows, Windows makes no distinction between "I failed to find the library you asked for", "I failed to find one of the libraries that your library asked for", or errors even further down (dependencies-of-dependencies-of-dependencies).

In my experience using BridgeStan however, there is usually one of two culprits. The first is that Windows is unable to find the mingw shared libraries (for example, libwinpthread). The second is if Windows is unable to locate TBB, which is in BRIDGESTAN_DIR/stan/lib/stan_math/lib/tbb. Adding one or both of these to your PATH environment variable may fix the issue. You can do this per-session in Python by using os.environ like os.environ['PATH'] = os.environ['PATH'] + ';' + full_path_to_library_directory

The MinGW distribution does supply a tool called ldd which can help debug this. If you can open a terminal and navigate to the folder containing the compiled model, ldd bernoulli_model.so will print out a list of libraries that it depends on, along with the location they were found in. If any of these are unknown (???), those are most likely the ones causing the error in Python.

dennis8 commented 1 year ago

Hi Ward,

Thanks for your reply. I'm trying to adjust my libraries based on your comment. I did the following:

For the TBB I could easily add the path from the stan_math subfolder to my environmental path variable. So this solved this issue. I had RTools 4.0 installed with cmdstanpy in my home directory in the subfolder '.cmdstan'. This is the mingw-make which I have been using so far. However in these subfolders there was nowhere a libwinpthread-1.dll anywhere. Therefore I installed RTools 4.3 and went through the following instructions in [https://mc-stan.org/docs/cmdstan-guide/cmdstan-installation.html#cpp-toolchain](cmdstan toolchain instruction). Removing the old path variables to Rtools 4.0 and adding the new ones to RTools 4.3 :
```
C:\rtools43\usr\bin
C:\rtools43\ucrt64\bin
```
Furthermore I installed all the requirements in the toolchain through msys2.exe
```
pacman -Sy mingw-w64-ucrt-x86_64-make mingw-w64-ucrt-x86_64-gcc
```
Within C:\rtools43\ucrt64\bin there is a libwinpthread-1.dll.

This is the output from Powershell that I get through the following commands:

where.exe libwinpthread-1.dll
> C:\rtools43\ucrt64\bin\libwinpthread-1.dll

ldd .\bernoulli_model.so
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffad8790000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffad6c80000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffad6150000)
        msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7ffad75b0000)
        USER32.dll => /c/WINDOWS/System32/USER32.dll (0x7ffad77e0000)
        win32u.dll => /c/WINDOWS/System32/win32u.dll (0x7ffad6430000)
        GDI32.dll => /c/WINDOWS/System32/GDI32.dll (0x7ffad7650000)
        gdi32full.dll => /c/WINDOWS/System32/gdi32full.dll (0x7ffad5ef0000)
        msvcp_win.dll => /c/WINDOWS/System32/msvcp_win.dll (0x7ffad60b0000)
        ucrtbase.dll => /c/WINDOWS/System32/ucrtbase.dll (0x7ffad66f0000)
        tbb.dll => /c/Users/MyUsername/.bridgestan/bridgestan-2.0.0/stan/lib/stan_math/lib/tbb/tbb.dll (0x676c0000)
        IMM32.DLL => /c/WINDOWS/System32/IMM32.DLL (0x7ffad8720000)
        tbbmalloc.dll => /c/Users/MyUsername/.bridgestan/bridgestan-2.0.0/stan/lib/stan_math/lib/tbb/tbbmalloc.dll (0x6a0000)
        tbbmalloc.dll => /c/Users/MyUsername/.bridgestan/bridgestan-2.0.0/stan/lib/stan_math/lib/tbb/tbbmalloc.dll (0x6a0000)

Also I tried running the following python file: [https://github.com/roualdes/bridgestan/blob/af2b73dc1b0e6e3c61fa6a1518d361fa6442efee/python/bridgestan/__init__.py](commit for windows checks), which doesn't result in going into any exceptions. But running a model still results in errors:

File "c:\bridgestan_test\example.py", line 20, in <module>
    model = bs.StanModel.from_stan_file(stan, data)
  File "C:\venv\lib\site-packages\bridgestan\model.py", line 244, in from_stan_file
    return cls(
  File "C:\venv\lib\site-packages\bridgestan\model.py", line 71, in __init__
    self.stanlib = ctypes.CDLL(self.lib_path)
  File "C:\Python\Python310\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'C:\Users\MyUsername\.bridgestan\bridgestan-2.0.0\test_models\bernoulli\bernoulli_model.so' (or one of its dependencies). Try using the full path with constructor syntax.

WardBrian commented 1 year ago

Did you delete the old file to make sure it re-built after changing the software you had installed?

WardBrian commented 1 year ago

@Dennis8 - I was wondering if this problem is still persisting after updating to BridgeStan 2.1.1?

WardBrian commented 1 year ago

Please re-open if the issue persists. Thank you!

roualdes / bridgestan

Python ctypes.CDDL compiled binary loading error #117