search

JuliaPy / PyCall.jl

Package to call Python functions from the Julia language
MIT License
1.47k stars 187 forks source link

Portable Installation of `PyCall`, `Conda` and `PyPlot` #890

Open RoyiAvital opened 3 years ago

RoyiAvital commented 3 years ago

On its installation PyCall, Conda and PyPlot creates many files which embed absolute paths. See https://github.com/JuliaPy/PyCall.jl/issues/767, https://github.com/JuliaPy/Conda.jl/issues/179, etc... Also the file qt.conf on the Conda folder (Relates to PyPlot).

I wonder if:

  1. The files will be written using relative paths instead of absolute paths.
  2. In case (1) isn't achievable, could a function be added to PyCall, Conda and PyPlot to regenerate the files with the updated path?

Currently, the only option is to remove all of them, delete the folders (Including Conda), remove the precompilations files and reinstall.

stevengj commented 3 years ago

I think we could use relative paths for things in .julia at least?

RoyiAvital commented 3 years ago

@stevengj , That would be great. Will make things Portable.

In the meanwhile, any way to force regeneration of the files? Like a forced build() or something like that?

stevengj commented 3 years ago

You can certainly re-run build for each package…it should re-generate the files if they have changed.

RoyiAvital commented 3 years ago

Will build() also update qt.conf and Conda related settings files?

RoyiAvital commented 3 years ago

OK, Moving the .julia folder and rerunning Pkg.build("PyCall"), Pkg.build("Conda") and Pkg.build("PyPlot") didn't regenerate the configuration files with the updated paths.

So I think this also should be taken care of, that build() will make sure all configuration is regenerated.

RoyiAvital commented 3 years ago

How can one force PyPlot() to run what it is running on first use again? It seems that's the procedure which configures Qt.

heetbeet commented 3 years ago

At Juliawin we make some changes to the environment before running any of the underlying binaries. Currently it's a bit of a game of whack-a-mole to keep Julia portable, but in summary this is what we have done to achieve some level of portability:

Here is the full script (in Windows batch unfortunately): https://github.com/heetbeet/juliawin/blob/main/bin/activate-juliawin-portability.bat

These requirements change over time as Anaconda and PyCall+friends grow in complexity. Some of the portability issues comes from Anaconda directly, so that would be quite difficult to address that from the PyCall side.

From the perspective of Juliawin, it would already be quite the help if PyCall has the option of having relative file-paths after compilation. Rebuilding all the packages is the more expensive part of the portability optimization.