search

MilesCranmer / PySR

High-Performance Symbolic Regression in Python and Julia
https://astroautomata.com/PySR
Apache License 2.0
2.35k stars 211 forks source link

[BUG] could not load library "libgit2" #267

Closed MilesCranmer closed 1 year ago

MilesCranmer commented 1 year ago

I'm seeing this issue on Windows 10. Python 3.10, Julia 1.8.5. PySR 0.11.14.

python -c "import pysr; pysr.install()"

will result in:

  Memory: 11.99899673461914 GB (6281.5859375 MB free)
  Uptime: 2551.312 sec
  Load Avg:  0.0  0.0  0.0
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, haswell)
  Threads: 1 on 8 virtual cores
Environment:
  JULIA_PROJECT = @pysr-0.11.14
  HOMEDRIVE = C:
  HOMEPATH = \Users\Shadow
  PATH = C:\Users\Shadow\Documents\PySR\venv\Scripts;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\Git\cmd;C:\Users\Shadow\AppData\Local\Programs\Python\Python310\Scripts\;C:\Users\Shadow\AppData\Local\Programs\Python\Python310\;C:\Users\Shadow\AppData\Local\Microsoft\WindowsApps;;C:\Users\Shadow\AppData\Local\Programs\Microsoft VS Code\bin
  PATHEXT = .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC
  PSMODULEPATH = C:\Program Files\WindowsPowerShell\Modules;C:\Windows\system32\WindowsPowerShell\v1.0\Modules
  _OLD_VIRTUAL_PATH = C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\Git\cmd;C:\Users\Shadow\AppData\Local\Programs\Python\Python310\Scripts\;C:\Users\Shadow\AppData\Local\Programs\Python\Python310\;C:\Users\Shadow\AppData\Local\Microsoft\WindowsApps;;C:\Users\Shadow\AppData\Local\Programs\Microsoft VS Code\bin
[ Info: Julia executable: C:\Users\Shadow\.julia\juliaup\julia-1.8.5+0.x64.w64.mingw32\bin\julia.exe
[ Info: Trying to import PyCall...
┌ Info: PyCall is already installed and compatible with Python executable.
│
│ PyCall:
│     python: C:\Users\Shadow\Documents\PySR\venv\Scripts\python.exe
│     libpython: C:\Users\Shadow\AppData\Local\Programs\Python\Python310\python310.dll
│ Python:
│     python: C:\Users\Shadow\Documents\PySR\venv\Scripts\python.exe
└     libpython: C:\Users\Shadow\AppData\Local\Programs\Python\Python310\python310.dll
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\Shadow\Documents\PySR\pysr\julia_helpers.py", line 96, in install
    _add_sr_to_julia_project(Main, io_arg)
  File "C:\Users\Shadow\Documents\PySR\pysr\julia_helpers.py", line 258, in _add_sr_to_julia_project
    Main.eval(f"Pkg.add([sr_spec, clustermanagers_spec], {io_arg})")
  File "C:\Users\Shadow\Documents\PySR\venv\lib\site-packages\julia\core.py", line 627, in eval
    ans = self._call(src)
  File "C:\Users\Shadow\Documents\PySR\venv\lib\site-packages\julia\core.py", line 555, in _call
    self.check_exception(src)
  File "C:\Users\Shadow\Documents\PySR\venv\lib\site-packages\julia\core.py", line 609, in check_exception
    raise JuliaError(u'Exception \'{}\' occurred while calling julia code:\n{}'
julia.core.JuliaError: Exception 'could not load library "libgit2"
The specified module could not be found. ' occurred while calling julia code:
Pkg.add([sr_spec, clustermanagers_spec], io=stderr)

I have installed Julia 1.8.5 using winget (recommended by juliaup). However, it seems like perhaps it does not find libgit2, even though I am able to install git packages via Julia REPL?

MilesCranmer commented 1 year ago

I was able to fix this by uninstalling the winget version, and just installing the official binary from the Julia website.

sakimarquis commented 1 year ago

I met the same problem in windows 11. And after unstalling juliaup and reinstalling the official Julia binary. I got a new error message. I can't solve it by installing SymbolicRegression.jl manually.

I solved this by installing in wsl. 

Julia Version 1.9.3
Commit bed2cd540a (2023-08-24 14:43 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
      Microsoft Windows [Version 10.0.22621.2361]
  CPU: 12th Gen Intel(R) Core(TM) i9-12900F:
                 speed         user         nice          sys         idle          irq
       #1-24  2419 MHz    8294552            0     23827319    1985108785       330678  ticks
  Memory: 63.797027587890625 GB (44775.7265625 MB free)
  Uptime: 84051.562 sec
  Load Avg:  0.0  0.0  0.0
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, alderlake)
  Threads: 1 on 24 virtual cores
Environment:
  JULIA_PROJECT = @pysr-0.16.3
  CHOCOLATEYLASTPATHUPDATE = "133202940231174213"
  CUDA_PATH = C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7
  CUDA_PATH_V11_7 = C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7
  HOMEDRIVE = P:
  HOMEPATH = \
  HOMESHARE = \\mbb\psychdept\users\hdx
  NVTOOLSEXT_PATH = C:\Program Files\NVIDIA Corporation\NvToolsExt\
  PATH = C:\Users\hdx\AppData\Local\miniconda3\envs\pysr;C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\Library\mingw-w64\bin;C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\Library\usr\bin;C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\Library\bin;C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\Scripts;C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\bin;C:\Users\hdx\AppData\Local\miniconda3\condabin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\libnvvp;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\dotnet;C:\Program Files\MATLAB\R2022b\bin;C:\Program Files\Microsoft VS Code\bin;C:\ProgramData\chocolatey\bin;C:\Program Files\NVIDIA Corporation\Nsight Compute 2022.2.0;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Anaconda3;C:\Anaconda3\Scripts;C:\Program Files\Git\cmd;C:\Program Files\PowerShell\7;C:\Users\hdx\AppData\Local\Microsoft\WindowsApps;C:\Users\hdx\AppData\Local\GitHubDesktop\bin;C:\Users\hdx\Julia\bin
  PATHEXT = .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.CPL
  PSMODULEPATH = C:\Users\hdx\Documents\WindowsPowerShell\Modules;C:\Program Files\WindowsPowerShell\Modules;C:\Windows\system32\WindowsPowerShell\v1.0\Modules
[ Info: Julia executable: C:\Users\hdx\Julia\bin\julia.exe
[ Info: Trying to import PyCall...
┌ Info: PyCall is already installed and compatible with Python executable.
│
│ PyCall:
│     python: C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\python.exe
│     libpython: C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\python310.dll
│ Python:
│     python: C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\python.exe
└     libpython: C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\python310.dll
    Updating registry at `C:\Users\hdx\.julia\registries\General.toml`
     Cloning git-repo `https://github.com/MilesCranmer/SymbolicRegression.jl`
Traceback (most recent call last):
  File "C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\lib\site-packages\pysr\__main__.py", line 4, in <module>
    _cli(prog_name="pysr")
  File "C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\lib\site-packages\click\core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\lib\site-packages\click\core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\lib\site-packages\click\core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\lib\site-packages\click\core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\lib\site-packages\click\core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\lib\site-packages\pysr\_cli\main.py", line 37, in _install
    install(julia_project, quiet, precompile)
  File "C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\lib\site-packages\pysr\julia_helpers.py", line 109, in install
    _add_sr_to_julia_project(Main, io_arg)
  File "C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\lib\site-packages\pysr\julia_helpers.py", line 272, in _add_sr_to_julia_project
    Main.eval(f"Pkg.add([sr_spec, clustermanagers_spec], {io_arg})")
  File "C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\lib\site-packages\julia\core.py", line 627, in eval
    ans = self._call(src)
  File "C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\lib\site-packages\julia\core.py", line 555, in _call
    self.check_exception(src)
  File "C:\Users\hdx\AppData\Local\miniconda3\envs\pysr\lib\site-packages\julia\core.py", line 609, in check_exception
    raise JuliaError(u'Exception \'{}\' occurred while calling julia code:\n{}'
julia.core.JuliaError: Exception 'failed to clone from https://github.com/MilesCranmer/SymbolicRegression.jl, error: GitError(Code:ERROR, Class:SSL, Your Julia is built with a SSL/TLS engine that libgit2 doesn't know how to configure to use a file or directory of certificate authority roots, but your environment specifies one via the SSL_CERT_FILE variable. If you believe your system's root certificates are safe to use, you can `export JULIA_SSL_CA_ROOTS_PATH=""` in your environment to use those instead.)' occurred while calling julia code:
Pkg.add([sr_spec, clustermanagers_spec], io=stderr)
MilesCranmer commented 1 year ago

Your issue might be different. It says:

Exception 'failed to clone from https://github.com/MilesCranmer/SymbolicRegression.jl, error: GitError(Code:ERROR, Class:SSL, Your Julia is built with a SSL/TLS engine that libgit2 doesn't know how to configure to use a file or directory of certificate authority roots, but your environment specifies one via the SSL_CERT_FILE variable. If you believe your system's root certificates are safe to use, you can export JULIA_SSL_CA_ROOTS_PATH="" in your environment to use those instead.)'

So it sounds like it can find libgit2 , but libgit2 is incompatible with the SSL engine used in Julia. So for this issue I might raise it on the Julia forums? https://discourse.julialang.org

I think you could get around this by running the following in the Julia REPL. First, hit ] to turn on Pkg mode. Then,

activate @pysr-0.16.3
rm SymbolicRegression ClusterManagers DynamicExpressions
add SymbolicRegression ClusterManagers DynamicExpressions

This should hopefully install the versions from the Julia registry, avoiding the GitHub repo. Replace 0.16.3 with whatever pysr version it is.

If you are using an older PySR version you might need to specify the version of SymbolicRegression.jl as well. (I think it's SymbolicRegression==0.22.4).

mariapintos commented 8 months ago

Hello Miles, the Julia forum woudlnt let me reply to the thread anymore, since I am new to the community and already exhausted the number of replies. PySR is already running thanks to this

import os
os.environ["JULIA_SSL_CA_ROOTS_PATH"] = ""

import pysr
pysr.install()  # Only for PySR < v0.17

Thank you so much for your help, really appreciated!