Installing Conda into a global non-writeable depot

[vchuravy]

We run into this issue at our local supercomputer where we would like to provide a pre-seeded depot that contains common pkg operations to reduce duplication.

Given a setup like:

mkdir /tmp/depot1
mkdir /tmp/depot2
mkdir /tmp/global

sudo mount -o bind,ro /tmp/depot2 /tmp/global

export JULIA_DEPOT_PATH="/tmp/depot2"
julia --project=. -e 'using Pkg; Pkg.add("Conda")'

Firstly ]build Conda fails since we are trying to touch a file in to non-writeable depot

export JULIA_DEPOT_PATH="/tmp/depot1:/tmp/global"
julia --project=. -e 'using Pkg; Pkg.build("Conda")'

Secondly adding a Conda package also fails since during the build we cached the path to the depot we got installed into.

export JULIA_DEPOT_PATH="/tmp/depot1:/tmp/global"
julia --project=. -e 'using Conda;Conda.add("Numpy")'

Any ideas for how we can make the user experience better here? @lmilechin

[stevengj]

You can pre-populate your depot with a file <depotpath>/conda/deps.jl that consists of

const ROOTENV = "/path/to/conda"
const MINICONDA_VERSION = "3"

If /path/to/conda is read-only, of course, users won't be able to do Conda.add, but I guess you could pre-install commonly used packages (e.g. numpy, matplotlib, …).

[vchuravy]

But https://github.com/JuliaPy/Conda.jl/blob/6deea1253a31facf80071279030444d564f2640a/src/Conda.jl#L21 pulls it deps file from the local installation instead of looking in <depotpath>/conda/deps.jl.

So we would also need to make sure that during building CONDA_JL_HOME is set to the right location. I think this runs into other issues as well since the build.jl presumes that the pathof(Conda) is writable.

[stevengj]

But https://github.com/JuliaPy/Conda.jl/blob/6deea1253a31facf80071279030444d564f2640a/src/Conda.jl#L21 pulls it deps file from the local installation instead of looking in /conda/deps.jl.

Hmm, right. I'm not sure why we did that? We should just change it to look in <depotpath>/conda/deps.jl since the two are identical.

Oh, I know: I guess I was worried about DEPOT_PATH changing at runtime compared to the build? Not sure if there is a better way to protect against that?

[vchuravy]

Oh, I know: I guess I was worried about DEPOT_PATH changing at runtime compared to the build? Not sure if there is a better way to protect against that?

What would happen is that the Conda.jl cachefile would invalidated and you recompile during loading. I think it should be fine to drop the deps.jl file.

[stevengj]

Okay, in that case a PR would be welcome.