Non Writable Conda directory in JuliaPro

[RoyiAvital]

My installation of Conda.jl fails on installing any package I try:

julia> Conda.add("scipy")
[ Info: Running `conda install -y scipy` in root environment
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: D:\Applications\Programming\JuliaProMKL\Profile\JuliaPro_v1.4.1-1\Conda_env

  added / updated specs:
    - scipy

The following NEW packages will be INSTALLED:

  scipy              pkgs/main/win-64::scipy-1.4.1-py37h9439919_0

Preparing transaction: done
Verifying transaction: \ WARNING conda.core.path_actions:verify(963): Unable to create environments file. Path not writable.
  environment location: C:\D:\Applications\Programming\JuliaProMKL\Profile\.conda\environments.txt

done
Executing transaction: done
Rolling back transaction: done

[Errno 22] Invalid argument: 'C:\\D:\\Applications\\Programming\\JuliaProMKL\\Profile\\.conda\\environments.txt'
()

ERROR: failed process: Process(setenv(`'D:\Applications\Programming\JuliaProMKL\Profile\JuliaPro_v1.4.1-1\Conda_env\Scripts\conda.exe' install -y scipy`,["PATH=D:\\Applications\\Programming\\JuliaProMKL\\Profile\\JuliaPro_v1.4.1-1\\Conda_env\\Library\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files\\MATLAB\\R2020a\\runtime\\win64;C:\\Program Files\\MATLAB\\R2020a\\bin;C:\\Program Files\\MATLAB\\R2019b\\runtime\\win64;C:\\Program Files\\MATLAB\\R2019b\\bin;C:\\Users\\Royi Avital\\AppData\\Local\\Microsoft\\WindowsApps;", "USERDOMAIN_ROAMINGPROFILE=DESKTOP-IIOBIL8", "HOMEPATH=D:\\Applications\\Programming\\JuliaProMKL\\Profile", "PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC", "NODE_ENV=production", "SESSIONNAME=Console", "SYSTEMROOT=C:\\WINDOWS", "APPDATA=D:\\Applications\\Programming\\JuliaProMKL\\Profile\\AppData", "PSMODULEPATH=C:\\Program Files\\WindowsPowerShell\\Modules;C:\\WINDOWS\\system32\\WindowsPowerShell\\v1.0\\Modules", "COMMONPROGRAMW6432=C:\\Program Files\\Common Files"  …  "LOGONSERVER=\\\\DESKTOP-IIOBIL8", "DRIVERDATA=C:\\Windows\\System32\\Drivers\\DriverData", "CONDA_PREFIX=D:\\Applications\\Programming\\JuliaProMKL\\Profile\\JuliaPro_v1.4.1-1\\Conda_env", "ATOM_HOME=D:\\Applications\\Programming\\JuliaProMKL\\.atom", "FPS_BROWSER_USER_PROFILE_STRING=Default", "SYSTEMDRIVE=C:", "FPS_BROWSER_APP_PROFILE_STRING=Internet Explorer", "PROCESSOR_ARCHITECTURE=AMD64", "JULIPRO_HOME=\"D:\\Applications\\Programming\\JuliaProMKL\\Profile\\.juliapro\\JuliaPro_v1.4.1-1\"", "OPENBLAS_MAIN_FREE=1"]), ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at .\process.jl:525 [inlined]
 [2] run(::Cmd; wait::Bool) at .\process.jl:440
 [3] run at .\process.jl:438 [inlined]
 [4] runconda(::Cmd, ::String) at D:\Applications\Programming\JuliaProMKL\Profile\.juliapro\JuliaPro_v1.4.1-1\packages\Conda\3rPhK\src\Conda.jl:114
 [5] add(::String, ::String; channel::String) at D:\Applications\Programming\JuliaProMKL\Profile\.juliapro\JuliaPro_v1.4.1-1\packages\Conda\3rPhK\src\Conda.jl:188
 [6] add at D:\Applications\Programming\JuliaProMKL\Profile\.juliapro\JuliaPro_v1.4.1-1\packages\Conda\3rPhK\src\Conda.jl:187 [inlined] (repeats 2 times)
 [7] top-level scope at none:0

Something with the paths doesn't work.

The strange thing (Is it?), installing PyPlot.jl worked. I had this issue with PyCall - https://github.com/JuliaPy/PyCall.jl/issues/767 if it adds information.

[RoyiAvital]

@stevengj , Any information missing in my report?

[stevengj]

The Unable to create environments file. Path not writable. error seems significant here.

Looks like C:\D:\Applications\Programming\JuliaProMKL\Profile\.conda is not writable? Potentially this a problem with JuliaPro…

[stevengj]

@ViralBShah, it seems like JuliaPro may be shipping with a Conda configured to work with a non-writable directory?

[RoyiAvital]

What makes it do something like C:\D:\Applications\Programming\JuliaProMKL\Profile\.conda?

It seems it mixed 2 different drive letters.

[RoyiAvital]

I think I have a clue what's going on. Could it be that when installing JuliaPro some paths are hard coded using the installation path? So if someone installs something on C:\ then move the packages to another drive (Updating the environment variables accordingly) yet some paths are hard coded it won't work.

In the case above, it seems Julia is using the drive with environment variable (Mixing them both).

@stevengj , Could it be that at first installation Conda.jl is writing paths somewhere so next time it is used it looks for those paths instead of system variables?

[stevengj]

Conda.jl saves its settings in joinpath(first(DEPOT_PATH), "conda", "deps.jl") (though you can overwrite it via the CONDA_JL_HOME environment variable). So if JuliaPro ships with this file indicating a non-writable root directory it will be a problem.

[ViralBShah]

Cc @aviks here.

[stevengj]

We install in abspath(first(DEPOT_PATH), "conda"), which is normally ~/.julia/conda, but maybe first(DEPOT_PATH) is the wrong thing in general? In any case, we don't default to any .conda directory IIRD, so it sounds like JuliaPro may be doing something special here.

[aviks]

I tried instaling a Conda package in a clean version of JuliaPro, and it succeeded. JuliaPro does not ship with incorrect configuration.

julia> Conda.add("numpy")
[ Info: Downloading miniconda installer ...
[ Info: Installing miniconda ...
[ Info: Running `conda config --add channels defaults --file 'C:\Users\myuser\.juliapro\JuliaPro_v1.3.1-1\conda\3\condarc-julia.yml' --force` in root environment
Warning: 'defaults' already in 'channels' list, moving to the top
[ Info: Running `conda update -y conda` in root environment
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: C:\Users\myuser\.juliapro\JuliaPro_v1.3.1-1\conda\3

  added / updated specs:
    - conda
.....

It seems the DEPOT_PATH has been manually changed in this case. Changing the DEPOT_PATH is possible in Julia or JuliaPro, but everything needs to be changed in co-ordination, which is somewhat involved.

[RoyiAvital]

@aviks, The question is, are some absolute paths written on installation? Maybe on first use? Otherwise it is a little strange it mixed 2 different drive letters.