JuliaError: Exception 'Unsatisfiable requirements detected for package DynamicExpressions / prior installation with conda

[dbl001]

Describe the bug

JuliaError: Exception 'Unsatisfiable requirements detected for package DynamicExpressions [a40a106e]:
 DynamicExpressions [a40a106e] log:
 ├─DynamicExpressions [a40a106e] has no known versions!
 └─restricted to versions 0.4 by SymbolicRegression [8254be44] — no versions left
   └─SymbolicRegression [8254be44] log:
     ├─possible versions are: 0.14.4 or uninstalled
     └─SymbolicRegression [8254be44] is fixed to version 0.14.4' occurred while calling julia code:
Pkg.add([sr_spec, clustermanagers_spec], io=stderr)

Version (please include the following information): MacOS Ventura 13.0.1 (22A400)

• Julia version [Run julia --version in the terminal]

• julia --version julia version 1.8.3

• Python version [Run python --version in the terminal]

• Python 3.8.13

• Did you install with pip or conda?

• pip

• 
  $ conda list pysr
  # packages in environment at /Users/davidlaxer/anaconda3/envs/ai:
  #
  # Name                    Version                   Build  Channel
  pysr                      0.11.11                  pypi_0    pypi

% pip show pysr Name: pysr Version: 0.11.11 Summary: Simple and efficient symbolic regression Home-page: https://github.com/MilesCranmer/pysr Author: Miles Cranmer Author-email: miles.cranmer@gmail.com License: Location: /Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages Requires: julia, numpy, pandas, scikit-learn, sympy Required-by:

- PySR version [Run `python -c 'import pysr; print(pysr.__version__)'`]
- 0.9.1
- Does the bug still appear with the latest version of PySR?

**Configuration**
- What are your PySR settings?
- What dataset are you running on?
- If possible, please share a minimal code example that produces the error.

**Error message**
Add the error message here, or whatever other information would be useful for debugging.

If the error is "Couldn't find equation file...", this error indicates something
went wrong with the backend. Please scroll up and copy
the output of Julia, rather than the output of python.

**Additional context**
Add any other context about the problem here.

Julia Version 1.8.3 Commit 0434deb161e (2022-11-14 20:14 UTC) Platform Info: OS: macOS (x86_64-apple-darwin21.4.0) uname: Darwin 22.1.0 Darwin Kernel Version 22.1.0: Sun Oct 9 20:14:54 PDT 2022; root:xnu-8792.41.9~2/RELEASE_X86_64 x86_64 i386 CPU: Intel(R) Core(TM) i7-10700K CPU @ 3.80GHz: speed user nice sys idle irq

1-16 3800 MHz 7543546 s 0 s 3955434 s 72076495 s 0 s

Memory: 128.0 GB (32470.4921875 MB free) Uptime: 951050.0 sec Load Avg: 8.20068359375 5.13525390625 4.3212890625 WORD_SIZE: 64 LIBM: libopenlibm LLVM: libLLVM-13.0.1 (ORCJIT, skylake) Threads: 1 on 16 virtual cores Environment: JULIA_DEPOT_PATH_BACKUP = JULIA_PROJECT_BACKUP = JULIA_LOAD_PATH_BACKUP = JULIA_DEPOT_PATH = /Users/davidlaxer/anaconda3/envs/ai/share/julia: JULIA_SSL_CA_ROOTS_PATH_BACKUP = JULIA_SSL_CA_ROOTS_PATH = JULIA_PROJECT = @pysr-0.11.11 TERM = xterm-color PATH = /Users/davidlaxer/.opam/coq-platform.2021.02.1/bin:/Users/davidlaxer/.juliaup/bin:/Users/davidlaxer/.cabal/bin:/Users/davidlaxer/.ghcup/bin:/Users/davidlaxer/anaconda3/envs/ai/bin:/Users/davidlaxer/anaconda3/condabin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/Users/davidlaxer/.cargo/bin:/Users/jetbrains/.local/bin XPC_FLAGS = 0x0 HOME = /Users/davidlaxer JAVA_HOME = :- JAVA_LD_LIBRARY_PATH = :- CAML_LD_LIBRARY_PATH = /Users/davidlaxer/.opam/coq-platform.2021.02.1/lib/stublibs:/Users/davidlaxer/.opam/coq-platform.2021.02.1/lib/ocaml/stublibs:/Users/davidlaxer/.opam/coq-platform.2021.02.1/lib/ocaml OCAML_TOPLEVEL_PATH = /Users/davidlaxer/.opam/coq-platform.2021.02.1/lib/toplevel PKG_CONFIG_PATH = /Users/davidlaxer/.opam/coq-platform.2021.02.1/lib/pkgconfig: CONDA_BACKUP_FFLAGS = -march=nocona -mtune=core2 -ftree-vectorize -fPIC -fstack-protector -O2 -pipe CONDA_BACKUP_FORTRANFLAGS = -march=nocona -mtune=core2 -ftree-vectorize -fPIC -fstack-protector -O2 -pipe CONDA_BACKUP_DEBUG_FFLAGS = -march=nocona -mtune=core2 -ftree-vectorize -fPIC -fstack-protector -O2 -pipe CONDA_BACKUP_DEBUG_FORTRANFLAGS = -march=nocona -mtune=core2 -ftree-vectorize -fPIC -fstack-protector -O2 -pipe [ Info: Julia version info [ Info: Julia executable: /Users/davidlaxer/anaconda3/envs/ai/share/julia/juliaup/julia-1.8.3+0.x64/bin/julia [ Info: Trying to import PyCall... ┌ Info: PyCall is already installed and compatible with Python executable. │ │ PyCall: │ python: /Users/davidlaxer/anaconda3/envs/ai/bin/python │ libpython: /Users/davidlaxer/anaconda3/envs/ai/lib/libpython3.8.dylib │ Python: │ python: /Users/davidlaxer/anaconda3/envs/ai/bin/python └ libpython: Resolving package versions...

JuliaError Traceback (most recent call last) Input In [5], in <cell line: 4>() 1 get_ipython().system('export JULIA_SSL_CA_ROOTS_PATH=""') 2 import pysr ----> 4 pysr.install()

File ~/anaconda3/envs/ai/lib/python3.8/site-packages/pysr/julia_helpers.py:87, in install(julia_project, quiet) 83 io_arg = _get_io_arg(quiet) 85 if is_shared: 86 # Install SymbolicRegression.jl: ---> 87 _add_sr_to_julia_project(Main, io_arg) 89 Main.eval("using Pkg") 90 Main.eval(f"Pkg.instantiate({io_arg})")

File ~/anaconda3/envs/ai/lib/python3.8/site-packages/pysr/julia_helpers.py:240, in _add_sr_to_julia_project(Main, io_arg) 230 Main.sr_spec = Main.PackageSpec( 231 name="SymbolicRegression", 232 url="https://github.com/MilesCranmer/SymbolicRegression.jl", 233 rev="v" + __symbolic_regression_jl_version__, 234 ) 235 Main.clustermanagers_spec = Main.PackageSpec( 236 name="ClusterManagers", 237 url="https://github.com/JuliaParallel/ClusterManagers.jl", 238 rev="14e7302f068794099344d5d93f71979aaf4fbeb3", 239 ) --> 240 Main.eval(f"Pkg.add([sr_spec, clustermanagers_spec], {io_arg})")

File ~/anaconda3/envs/ai/lib/python3.8/site-packages/julia/core.py:627, in Julia.eval(self, src) 625 if src is None: 626 return None --> 627 ans = self._call(src) 628 if not ans: 629 return None

File ~/anaconda3/envs/ai/lib/python3.8/site-packages/julia/core.py:555, in Julia._call(self, src) 553 # logger.debug("_call(%s)", src) 554 ans = self.api.jl_eval_string(src.encode('utf-8')) --> 555 self.check_exception(src) 557 return ans

File ~/anaconda3/envs/ai/lib/python3.8/site-packages/julia/core.py:609, in Julia.check_exception(self, src) 607 else: 608 exception = sprint(showerror, self._as_pyobj(res)) --> 609 raise JuliaError(u'Exception \'{}\' occurred while calling julia code:\n{}' 610 .format(exception, src))

JuliaError: Exception 'Unsatisfiable requirements detected for package DynamicExpressions [a40a106e]: DynamicExpressions [a40a106e] log: ├─DynamicExpressions [a40a106e] has no known versions! └─restricted to versions 0.4 by SymbolicRegression [8254be44] — no versions left └─SymbolicRegression [8254be44] log: ├─possible versions are: 0.14.4 or uninstalled └─SymbolicRegression [8254be44] is fixed to version 0.14.4' occurred while calling julia code: Pkg.add([sr_spec, clustermanagers_spec], io=stderr)

% julia () | Documentation: https://docs.julialang.org () | () () | | |_ | Type "?" for help, "]?" for Pkg help. | | | | | | |/ ` | | | | || | | | (| | | Version 1.8.3 (2022-11-14) / |_'|||_'_| | Official https://julialang.org/ release |/ |

julia> using Pkg

julia> Pkg.add("DynamicExpressions") ERROR: The following package names could not be resolved:

• DynamicExpressions (not found in project, manifest or registry) Stacktrace: [1] pkgerror(msg::String) @ Pkg.Types ~/anaconda3/envs/ai/share/julia/juliaup/julia-1.8.3+0.x64/share/julia/stdlib/v1.8/Pkg/src/Types.jl:67 [2] ensure_resolved(ctx::Pkg.Types.Context, manifest::Pkg.Types.Manifest, pkgs::Vector{Pkg.Types.PackageSpec}; registry::Bool) @ Pkg.Types ~/anaconda3/envs/ai/share/julia/juliaup/julia-1.8.3+0.x64/share/julia/stdlib/v1.8/Pkg/src/Types.jl:952 [3] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform, kwargs::Base.Pairs{Symbol, Base.TTY, Tuple{Symbol}, NamedTuple{(:io,), Tuple{Base.TTY}}}) @ Pkg.API ~/anaconda3/envs/ai/share/julia/juliaup/julia-1.8.3+0.x64/share/julia/stdlib/v1.8/Pkg/src/API.jl:264 [4] add(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}) @ Pkg.API ~/anaconda3/envs/ai/share/julia/juliaup/julia-1.8.3+0.x64/share/julia/stdlib/v1.8/Pkg/src/API.jl:156 [5] add(pkgs::Vector{Pkg.Types.PackageSpec}) @ Pkg.API ~/anaconda3/envs/ai/share/julia/juliaup/julia-1.8.3+0.x64/share/julia/stdlib/v1.8/Pkg/src/API.jl:145 [6] #add#27 @ ~/anaconda3/envs/ai/share/julia/juliaup/julia-1.8.3+0.x64/share/julia/stdlib/v1.8/Pkg/src/API.jl:144 [inlined] [7] add @ ~/anaconda3/envs/ai/share/julia/juliaup/julia-1.8.3+0.x64/share/julia/stdlib/v1.8/Pkg/src/API.jl:144 [inlined] [8] #add#26 @ ~/anaconda3/envs/ai/share/julia/juliaup/julia-1.8.3+0.x64/share/julia/stdlib/v1.8/Pkg/src/API.jl:143 [inlined] [9] add(pkg::String) @ Pkg.API ~/anaconda3/envs/ai/share/julia/juliaup/julia-1.8.3+0.x64/share/julia/stdlib/v1.8/Pkg/src/API.jl:143 [10] top-level scope @ REPL[2]:1

julia>

The code works properly on Google CoLab.

[MilesCranmer]

Hi @dbl001,

It looks like your Julia package registry does not contain a dependency that SymbolicRegression.jl needs. The conda version of PySR ships its own fixed Julia package registry (the "DEPOT"), so it seems like maybe an older version of PySR is still being used for Julia packages, rather than the latest one. (The pip version of PySR just uses a standard Julia package registry which automatically updates.)

I have a couple questions to help triage this.

1. I see you have both pip and conda set up. Did you install PySR with both? Can you uninstall PySR from everything, for all versions (i.e., keep running pip uninstall pysr and similar with conda until it is out of versions), and then trying installing it in just one of them?
2. Why does it say you have PySR 0.9.1 when you do python -c 'import pysr; print(pysr.__version__)', but then your conda and pip show it as PySR 0.11.11? Maybe try uninstalling all versions of PySR so you can be certain the error is from the current version?

Cheers, Miles

[dbl001]

Conda show 'pysr' was install from pipy:

% conda list pysr
# packages in environment at /Users/davidlaxer/anaconda3:
#
# Name                    Version                   Build  Channel
pysr                      0.9.1                    pypi_0    pypi

I also have source for: SymbolicRegression.jl pysr

No doubt I ran 'python setup.py install', etc.

[dbl001]

Same error after uninstalling 'pysr' with pip a few times.

% conda list pysr
# packages in environment at /Users/davidlaxer/anaconda3:
#
# Name                    Version                   Build  Channel
pysr                      0.9.1                    pypi_0    pypi
(base) davidlaxer@x86_64-apple-darwin13 ~ % pip uninstall  pysr
Found existing installation: pysr 0.9.1
Uninstalling pysr-0.9.1:
  Would remove:
    /Users/davidlaxer/anaconda3/lib/python3.8/site-packages/datasets/FeynmanEquations.csv
    /Users/davidlaxer/anaconda3/lib/python3.8/site-packages/pysr-0.9.1.dist-info/*
    /Users/davidlaxer/anaconda3/lib/python3.8/site-packages/pysr/*
    /Users/davidlaxer/anaconda3/lib/python3.8/site-packages/test/*
Proceed (Y/n)? y
  Successfully uninstalled pysr-0.9.1
(base) davidlaxer@x86_64-apple-darwin13 ~ % pip uninstall pysr
WARNING: Skipping pysr as it is not installed.
(base) davidlaxer@x86_64-apple-darwin13 ~ % conda list pysr
# packages in environment at /Users/davidlaxer/anaconda3:
#
# Name                    Version                   Build  Channel
(base) davidlaxer@x86_64-apple-darwin13 ~ % pip install pysr
Collecting pysr
  Using cached pysr-0.11.11-py3-none-any.whl (62 kB)
Requirement already satisfied: numpy in ./anaconda3/lib/python3.8/site-packages (from pysr) (1.22.3)
Requirement already satisfied: scikit-learn>=1.0.0 in ./anaconda3/lib/python3.8/site-packages (from pysr) (1.0.2)
Requirement already satisfied: sympy in ./anaconda3/lib/python3.8/site-packages (from pysr) (1.10.1)
Collecting julia==0.6.0
  Downloading julia-0.6.0-py2.py3-none-any.whl (68 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 68.6/68.6 kB 1.2 MB/s eta 0:00:00
Requirement already satisfied: pandas in ./.local/lib/python3.8/site-packages (from pysr) (1.3.4)
Requirement already satisfied: threadpoolctl>=2.0.0 in ./anaconda3/lib/python3.8/site-packages (from scikit-learn>=1.0.0->pysr) (2.2.0)
Requirement already satisfied: scipy>=1.1.0 in ./anaconda3/lib/python3.8/site-packages (from scikit-learn>=1.0.0->pysr) (1.7.3)
Requirement already satisfied: joblib>=0.11 in ./anaconda3/lib/python3.8/site-packages (from scikit-learn>=1.0.0->pysr) (1.1.0)
Requirement already satisfied: pytz>=2017.3 in ./anaconda3/lib/python3.8/site-packages (from pandas->pysr) (2022.1)
Requirement already satisfied: python-dateutil>=2.7.3 in ./anaconda3/lib/python3.8/site-packages (from pandas->pysr) (2.8.2)
Requirement already satisfied: mpmath>=0.19 in ./anaconda3/lib/python3.8/site-packages (from sympy->pysr) (1.2.1)
Requirement already satisfied: six>=1.5 in ./anaconda3/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas->pysr) (1.16.0)
Installing collected packages: julia, pysr
  Attempting uninstall: julia
    Found existing installation: julia 0.5.7
    Uninstalling julia-0.5.7:
      Successfully uninstalled julia-0.5.7
Successfully installed julia-0.6.0 pysr-0.11.11
(base) davidlaxer@x86_64-apple-darwin13 ~ % pip show pysr
Name: pysr
Version: 0.11.11
Summary: Simple and efficient symbolic regression
Home-page: https://github.com/MilesCranmer/pysr
Author: Miles Cranmer
Author-email: miles.cranmer@gmail.com
License: 
Location: /Users/davidlaxer/anaconda3/lib/python3.8/site-packages
Requires: julia, numpy, pandas, scikit-learn, sympy
Required-by: 
(base) davidlaxer@x86_64-apple-darwin13 ~ % 

[MilesCranmer]

Thanks. How are you installing Julia? It looks like the Julia install is inside conda. If so, conda uninstall julia, and then install Julia normally: https://julialang.org/downloads/.

[MilesCranmer]

Btw, one confusing thing: in conda, julia is the language julia, but in pip, julia is actually PyJulia (the interface)

[dbl001]

% conda list julia
# packages in environment at /Users/davidlaxer/anaconda3:
#
# Name                    Version                   Build  Channel
julia                     0.6.0                    pypi_0    pypi
(base) davidlaxer@x86_64-apple-darwin13 ~ % pip show julia
Name: julia
Version: 0.6.0
Summary: Julia/Python bridge with IPython support.
Home-page: http://julialang.org
Author: The Julia and IPython development teams.
Author-email: julia@julialang.org
License: MIT
Location: /Users/davidlaxer/anaconda3/lib/python3.8/site-packages
Requires: 
Required-by: pysr
(base) davidlaxer@x86_64-apple-darwin13 ~ % which julia
/Users/davidlaxer/.juliaup/bin/julia
(base) davidlaxer@x86_64-apple-darwin13 ~ % julia --version
The latest version of Julia in the `release` channel is 1.8.3+0.x64. You currently have `1.8.2+0.x64` installed. Run:

  juliaup update

to install Julia 1.8.3+0.x64 and update the `release` channel to that version.
julia version 1.8.2

[MilesCranmer]

So far so good. But I am confused why it lists your JULIA_DEPOT as if you had installed julia with conda:

Environment:
...
  JULIA_DEPOT_PATH = /Users/davidlaxer/anaconda3/envs/ai/share/julia:
...

maybe just try deleting share/julia and seeing if that fixes it?

[MilesCranmer]

For reference, here is mine (also installed with juliaup):

> julia
julia> DEPOT_PATH
3-element Vector{String}:
 "/Users/mcranmer/.julia"
 "/Users/mcranmer/.julia/juliaup/julia-1.8.3+0.aarch64/local/share/julia"
 "/Users/mcranmer/.julia/juliaup/julia-1.8.3+0.aarch64/share/julia"

so I would expect your DEPOT_PATH to be similar.

[dbl001]

I deleted share/julia in my Conda virtual environment

% julia
Error: The Julia launcher failed to figure out which juliaup channel to use.

% ipython
Python 3.8.13 (default, Mar 28 2022, 06:16:26) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.3.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import julia

In [2]: import pysr

In [3]: from julia import Julia

In [4]: julia = Julia(compiled_modules=False)
   ...: from julia import Main
   ...: from julia.tools import redirect_output_streams
   ...: 
   ...: redirect_output_streams()
---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
Input In [4], in <cell line: 1>()
----> 1 julia = Julia(compiled_modules=False)
      2 from julia import Main
      3 from julia.tools import redirect_output_streams

File ~/anaconda3/envs/ai/lib/python3.8/site-packages/julia/core.py:702, in LegacyJulia.__init__(self, *args, **kwargs)
    701 def __init__(self, *args, **kwargs):
--> 702     self.__julia = Julia(*args, **kwargs)

File ~/anaconda3/envs/ai/lib/python3.8/site-packages/julia/core.py:468, in Julia.__init__(self, init_julia, jl_init_path, runtime, jl_runtime_path, debug, **julia_options)
    466     self.api = get_libjulia()
    467 elif init_julia:
--> 468     jlinfo = JuliaInfo.load(runtime)
    469     if jlinfo.version_info < (0, 7):
    470         raise RuntimeError("PyJulia does not support Julia < 0.7 anymore")

File ~/anaconda3/envs/ai/lib/python3.8/site-packages/julia/juliainfo.py:87, in JuliaInfo.load(cls, julia, **popen_kwargs)
     83         raise subprocess.CalledProcessError(
     84             retcode, [julia, "-e", "..."], output
     85         )
     86     else:
---> 87         raise subprocess.CalledProcessError(
     88             retcode, [julia, "-e", "..."], stdout, stderr
     89         )
     91 stderr = stderr.strip()
     92 if stderr:

CalledProcessError: Command '['julia', '-e', '...']' returned non-zero exit status 1.

In [5]: 

In [5]: print(julia.__version__)
0.6.0

[MilesCranmer]

Okay so now that it's deleted, you should re-install PyCall.jl using python -c 'import pysr; pysr.install()'. Otherwise PyJulia can't access Julia yet.

Also - before that, I would try just running using Pkg; Pkg.add("DynamicExpressions") within Julia itself to make sure everything is good now.

[MilesCranmer]

Btw, the redirect_output_streams() is just a colab-specific hack. You don't need to do that in IPython.

[dbl001]

julia> DEPOT_PATH
3-element Vector{String}:
 "/Users/davidlaxer/.julia"
 "/Applications/Julia-1.8.app/Contents/Resources/julia/local/share/julia"
 "/Applications/Julia-1.8.app/Contents/Resources/julia/share/julia"

julia> using Pkg; Pkg.add("DynamicExpressions")
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
   Installed SLEEFPirates ───────────────────── v0.6.37
   Installed ArrayInterfaceOffsetArrays ─────── v0.1.7
   Installed SIMDTypes ──────────────────────── v0.1.0
   Installed ThreadingUtilities ─────────────── v0.5.0
   Installed SIMDDualNumbers ────────────────── v0.1.1
   Installed CpuId ──────────────────────────── v0.3.1
   Installed ManualMemory ───────────────────── v0.1.8
   Installed DynamicExpressions ─────────────── v0.4.2
   Installed OffsetArrays ───────────────────── v1.12.8
   Installed SnoopPrecompile ────────────────── v1.0.1
   Installed BitTwiddlingConvenienceFunctions ─ v0.1.5
   Installed LayoutPointers ─────────────────── v0.1.12
   Installed PolyesterWeave ─────────────────── v0.1.11
   Installed CloseOpenIntervals ─────────────── v0.1.11
   Installed CPUSummary ─────────────────────── v0.1.30
   Installed HostCPUFeatures ────────────────── v0.1.13
   Installed VectorizationBase ──────────────── v0.21.56
   Installed LoopVectorization ──────────────── v0.12.141
    Updating `~/.julia/environments/v1.8/Project.toml`
  [a40a106e] + DynamicExpressions v0.4.2
    Updating `~/.julia/environments/v1.8/Manifest.toml`
  [015c0d05] + ArrayInterfaceOffsetArrays v0.1.7
  [62783981] + BitTwiddlingConvenienceFunctions v0.1.5
  [2a0fbf3d] + CPUSummary v0.1.30
  [fb6a15b2] + CloseOpenIntervals v0.1.11
  [adafc99b] + CpuId v0.3.1
  [a40a106e] + DynamicExpressions v0.4.2
  [3e5b6fbb] + HostCPUFeatures v0.1.13
  [10f19ff3] + LayoutPointers v0.1.12
  [bdcacae8] + LoopVectorization v0.12.141
  [d125e4d3] + ManualMemory v0.1.8
  [6fe1bfb0] + OffsetArrays v1.12.8
  [1d0040c9] + PolyesterWeave v0.1.11
  [3cdde19b] + SIMDDualNumbers v0.1.1
  [94e857df] + SIMDTypes v0.1.0
  [476501e8] + SLEEFPirates v0.6.37
  [66db9d55] + SnoopPrecompile v1.0.1
  [8290d209] + ThreadingUtilities v0.5.0
  [3d5dd08c] + VectorizationBase v0.21.56
Precompiling project...
  185 dependencies successfully precompiled in 66 seconds

julia> 

% python -c 'import pysr; pysr.install()'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/davidlaxer/PySR/pysr/julia_helpers.py", line 76, in install
    _julia_version_assertion()
  File "/Users/davidlaxer/PySR/pysr/julia_helpers.py", line 251, in _julia_version_assertion
    if not is_julia_version_greater_eq(version=(1, 6, 0)):
  File "/Users/davidlaxer/PySR/pysr/julia_helpers.py", line 124, in is_julia_version_greater_eq
    juliainfo = _load_juliainfo()
  File "/Users/davidlaxer/PySR/pysr/julia_helpers.py", line 25, in _load_juliainfo
    juliainfo = JuliaInfo.load(julia="julia")
  File "/Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages/julia/juliainfo.py", line 87, in load
    raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['julia', '-e', '...']' returned non-zero exit status 1.
(ai) davidlaxer@x86_64-apple-darwin13 pysr % ipython
Python 3.8.13 (default, Mar 28 2022, 06:16:26) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.3.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pysr

In [2]: pysr.install()
---------------------------------------------------------------------------
CalledProcessError                        Traceback (most recent call last)
Input In [2], in <cell line: 1>()
----> 1 pysr.install()

File ~/PySR/pysr/julia_helpers.py:76, in install(julia_project, quiet)
     69 """
     70 Install PyCall.jl and all required dependencies for SymbolicRegression.jl.
     71 
     72 Also updates the local Julia registry.
     73 """
     74 import julia
---> 76 _julia_version_assertion()
     77 # Set JULIA_PROJECT so that we install in the pysr environment
     78 processed_julia_project, is_shared = _process_julia_project(julia_project)

File ~/PySR/pysr/julia_helpers.py:251, in _julia_version_assertion()
    250 def _julia_version_assertion():
--> 251     if not is_julia_version_greater_eq(version=(1, 6, 0)):
    252         raise NotImplementedError(
    253             "PySR requires Julia 1.6.0 or greater. "
    254             "Please update your Julia installation."
    255         )

File ~/PySR/pysr/julia_helpers.py:124, in is_julia_version_greater_eq(juliainfo, version)
    122 """Check if Julia version is greater than specified version."""
    123 if juliainfo is None:
--> 124     juliainfo = _load_juliainfo()
    125 current_version = (
    126     juliainfo.version_major,
    127     juliainfo.version_minor,
    128     juliainfo.version_patch,
    129 )
    130 return current_version >= version

File ~/PySR/pysr/julia_helpers.py:25, in _load_juliainfo()
     22 from julia.core import JuliaInfo
     24 try:
---> 25     juliainfo = JuliaInfo.load(julia="julia")
     26 except FileNotFoundError:
     27     env_path = os.environ["PATH"]

File ~/anaconda3/envs/ai/lib/python3.8/site-packages/julia/juliainfo.py:87, in JuliaInfo.load(cls, julia, **popen_kwargs)
     83         raise subprocess.CalledProcessError(
     84             retcode, [julia, "-e", "..."], output
     85         )
     86     else:
---> 87         raise subprocess.CalledProcessError(
     88             retcode, [julia, "-e", "..."], stdout, stderr
     89         )
     91 stderr = stderr.strip()
     92 if stderr:

CalledProcessError: Command '['julia', '-e', '...']' returned non-zero exit status 1.

[MilesCranmer]

Well, your Julia install looks like it's working now. It's just the interface between the two that has some problem... What is python -c 'import os; print(os.environ["PATH"])'? My suspicion is that the julia binary isn't on the PATH variable inside Python.

[dbl001]

% python -c 'import os; print(os.environ["PATH"])'
/Users/davidlaxer/.opam/_coq-platform_.2021.02.1/bin:/Users/davidlaxer/.juliaup/bin:/Users/davidlaxer/.cabal/bin:/Users/davidlaxer/.ghcup/bin:/Users/davidlaxer/anaconda3/envs/ai/bin:/Users/davidlaxer/anaconda3/condabin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/Users/davidlaxer/.cargo/bin:/Users/jetbrains/.local/bin

[MilesCranmer]

Looks fine to me... What does just running python -c 'from julia.core import JuliaInfo; juliainfo = JuliaInfo.load(julia="julia")' do? (i.e., what is the full error message)?

[dbl001]

% env | grep PATH
PATH=/Users/davidlaxer/.opam/_coq-platform_.2021.02.1/bin:/Users/davidlaxer/.juliaup/bin:/Users/davidlaxer/.cabal/bin:/Users/davidlaxer/.ghcup/bin:/Users/davidlaxer/anaconda3/envs/ai/bin:/Users/davidlaxer/anaconda3/condabin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/Users/davidlaxer/.cargo/bin:/Users/jetbrains/.local/bin
JAVA_LD_LIBRARY_PATH=:-
CAML_LD_LIBRARY_PATH=/Users/davidlaxer/.opam/_coq-platform_.2021.02.1/lib/stublibs:/Users/davidlaxer/.opam/_coq-platform_.2021.02.1/lib/ocaml/stublibs:/Users/davidlaxer/.opam/_coq-platform_.2021.02.1/lib/ocaml
OCAML_TOPLEVEL_PATH=/Users/davidlaxer/.opam/_coq-platform_.2021.02.1/lib/toplevel
PKG_CONFIG_PATH=/Users/davidlaxer/.opam/_coq-platform_.2021.02.1/lib/pkgconfig:
JULIA_DEPOT_PATH_BACKUP=
JULIA_LOAD_PATH_BACKUP=
JULIA_DEPOT_PATH=/Users/davidlaxer/anaconda3/envs/ai/share/julia:
JULIA_SSL_CA_ROOTS_PATH_BACKUP=
JULIA_SSL_CA_ROOTS_PATH=/Users/davidlaxer/anaconda3/envs/ai/ssl/cacert.pem
(ai) davidlaxer@x86_64-apple-darwin13 pysr % python -c 'from julia.core import JuliaInfo; juliainfo = JuliaInfo.load(julia="julia")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages/julia/juliainfo.py", line 87, in load
    raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['julia', '-e', '...']' returned non-zero exit status 1.

 % which julia
/Users/davidlaxer/.juliaup/bin/julia
(ai) davidlaxer@x86_64-apple-darwin13 pysr % whereis julia
julia: /Users/davidlaxer/.juliaup/bin/julia /Users/davidlaxer/anaconda3/envs/ai/share/man/man1/julia.1

% conda list julia
# packages in environment at /Users/davidlaxer/anaconda3/envs/ai:
#
# Name                    Version                   Build  Channel
julia                     1.6.5                h132cb31_2    conda-forge
pyjulia                   0.6.0              pyhd8ed1ab_0    conda-forge

[MilesCranmer]

Hmm, why would JULIA_DEPOT_PATH still be /Users/davidlaxer/anaconda3/envs/ai/share/julia if you have deleted it, and also successfully installed a Julia package to a different depot... I guess that's why it's erroring out.

I think your conda environment's activate script is setting JULIA_DEPOT_PATH maybe from an earlier install of pysr using conda. I guess when you uninstalled it, that JULIA_DEPOT_PATH was not unset. @mkitti we might want to fix this when julia is uninstalled from conda somehow.

The easy fix for you is to explicitly set export JULIA_DEPOT_PATH=/Applications/Julia-1.8.app/Contents/Resources/julia/share/julia from bash before starting python.

The longterm fix is to find the activate script in your conda environment, and see where it is setting JULIA_DEPOT_PATH, and then comment out that line. For safe measure do the same for JULIA_SSL_CA_ROOTS_PATH since it also points to your conda's old julia install.

[MilesCranmer]

In your updated comment, I think this:

% conda list julia
# packages in environment at /Users/davidlaxer/anaconda3/envs/ai:
#
# Name                    Version                   Build  Channel
julia                     1.6.5                h132cb31_2    conda-forge
pyjulia                   0.6.0              pyhd8ed1ab_0    conda-forge

indicates that Julia was not actually uninstalled from your conda env?

[MilesCranmer]

i.e., run conda uninstall julia pyjulia

[dbl001]

% conda list julia
# packages in environment at /Users/davidlaxer/anaconda3/envs/ai:
#
# Name                    Version                   Build  Channel
(ai) davidlaxer@x86_64-apple-darwin13 pysr % pip show julia
WARNING: Package(s) not found: julia

 % python -c 'from julia.core import JuliaInfo; juliainfo = JuliaInfo.load(julia="julia")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'julia'
(ai) davidlaxer@x86_64-apple-darwin13 pysr % ipython
Python 3.8.13 (default, Mar 28 2022, 06:16:26) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.3.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pysr
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Input In [1], in <cell line: 1>()
----> 1 import pysr

File ~/PySR/pysr/__init__.py:2, in <module>
      1 from .version import __version__
----> 2 from .sr import (
      3     pysr,
      4     PySRRegressor,
      5     best,
      6     best_tex,
      7     best_callable,
      8     best_row,
      9 )
     10 from .julia_helpers import install
     11 from .feynman_problems import Problem, FeynmanProblem

File ~/PySR/pysr/sr.py:24, in <module>
     18 from sklearn.utils import check_array, check_consistent_length, check_random_state
     19 from sklearn.utils.validation import (
     20     _check_feature_names_in,
     21     check_is_fitted,
     22 )
---> 24 from .julia_helpers import (
     25     init_julia,
     26     _process_julia_project,
     27     is_julia_version_greater_eq,
     28     _escape_filename,
     29     _load_cluster_manager,
     30     _update_julia_project,
     31     _load_backend,
     32 )
     33 from .export_numpy import CallableEquation
     34 from .export_latex import generate_single_table, generate_multiple_tables, to_latex

File ~/PySR/pysr/julia_helpers.py:7, in <module>
      5 from pathlib import Path
      6 import os
----> 7 from julia.api import JuliaError
      9 from .version import __version__, __symbolic_regression_jl_version__
     11 juliainfo = None

ModuleNotFoundError: No module named 'julia'

[dbl001]

So, manually delete this directory:

~/anaconda3/envs/ai/lib/python3.8/site-packages/julia

[dbl001]

Hmmm... Requirement already satisfied: julia==0.6.0 in /Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages (from pysr) (0.6.0)

% pip install pysr
Requirement already satisfied: pysr in /Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages (0.11.11)
Requirement already satisfied: pandas in /Users/davidlaxer/.local/lib/python3.8/site-packages (from pysr) (1.3.4)
Requirement already satisfied: julia==0.6.0 in /Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages (from pysr) (0.6.0)
Requirement already satisfied: sympy in /Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages (from pysr) (1.10.1)
Requirement already satisfied: scikit-learn>=1.0.0 in /Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages (from pysr) (1.0.2)
Requirement already satisfied: numpy in /Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages (from pysr) (1.22.3)
Requirement already satisfied: threadpoolctl>=2.0.0 in /Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages (from scikit-learn>=1.0.0->pysr) (2.2.0)
Requirement already satisfied: scipy>=1.1.0 in /Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages (from scikit-learn>=1.0.0->pysr) (1.8.1)
Requirement already satisfied: joblib>=0.11 in /Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages (from scikit-learn>=1.0.0->pysr) (1.1.0)
Requirement already satisfied: pytz>=2017.3 in /Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages (from pandas->pysr) (2022.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages (from pandas->pysr) (2.8.2)
Requirement already satisfied: mpmath>=0.19 in /Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages (from sympy->pysr) (1.2.1)
Requirement already satisfied: six>=1.5 in /Users/davidlaxer/anaconda3/envs/ai/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas->pysr) (1.16.0)
WARNING: There was an error checking the latest version of pip.
(ai) davidlaxer@x86_64-apple-darwin13 pysr % 

[MilesCranmer]

Looks a bit tricky to debug that, it seems like it's an environment issue. Maybe try to just create a new conda environment? Conda is tricky to fix when you combine pip installs with conda installs.

Worst case you could always install the docker, which is pretty easy, docker pull ghcr.io/milescranmer/pysr and docker run -it --rm -v "$PWD":/workspace ghcr.io/milescranmer/pysr ipython - that IPython should be ready to go with pysr, and you can access your current directory inside the container in the /workspace folder.

[mkitti]

I recommend just starting a fresh environment first.

mamba create -n pysr_test -c conda-forge pysr julia

Replace mamba with conda if you are not using that already.

[dbl001]

It's working on MacOS in a new Conda virtual environment. Thanks for your help!

[MilesCranmer]

Awesome. Thanks @mkitti!