juliaup in a multi-user environment

[dmbates]

We would like to install Juliaup as root on a departmental Linux cluster and have it available to users. I haven't found any documentation specific to this use. I presume we should use a custom installation but I am not entirely clear what directories would go where. For a personal installation the are both ~/.juliaup and ~/.julia/juliaup directories created. What would be system-wide equivalents?

[johnnychen94]

Installs in /opt/juliaup and make a symlink to /usr/local/bin/juliaup? But for juliaup there might be a lot of permission issues if installed in a shared folder unless we keep personal configs somewhere in $HOME.

[davidanthoff]

So just as a warning, I think that scenario is not built out yet... But we should!

Also, we are going to make one more change to the default setup of juliaup so that ~/.juliaup will go away.

I think there are two scenarios: a) juliaup itself is installed system-wide, but each user has their own JULIA_DEPOT or b) a scenario where both juliaup and the Julia depot are meant to be system wide.

I think a) might sort-of work today, as long as you don't expect any of the self-update stuff for juliaup to work. You essentially just need to put the juliaup and julialauncher binary into a non-user specific folder that is on the PATH, then also make sure that you have a symlink from julia to julialauncher on the PATH, and I think then things should more or less work. EXCEPT, that juliaup will not auto-update, so you will have to handle that yourself...

[dmbates]

Thanks for the replies @johnnychen94 and @davidanthoff . Can you point me to some information on JULIA_DEPOT, David? I should know what that environment variable refers to but I don't. Essentially I want users to share the installed versions of Julia and the multiplexer and its configuration. I would expect that the SysAdmin would need to do the installations of new versions but I hope that a non-privileged user could check

juliaup st

to find what versions are currently installed.

[dmbates]

I found the description of JULIA_DEPOT in https://docs.julialang.org/en/v1/manual/environment-variables/

[davidanthoff]

Well, first I used the wrong name, the env var is called JULIA_DEPOT_PATH, see https://docs.julialang.org/en/v1/manual/environment-variables/#JULIA_DEPOT_PATH.

So, I think you are interested in some kind of variant of b). In your system, do you want users to have the ability to install packages, or is that also meant to be systemwide?

At the moment, juliaup treats individual Julia versions as if they were a bit like packages...

[dmbates]

I'm fine with individual users having their own package repositories as it quickly becomes too much of a headache to try to maintain a system-wide package repository.

I guess I would prefer if the installations of julia versions were shared. If they are installed for each user then the users could just install their own version of juliaup. Each installation of a Julia version weighs in at 400-500 MB and we would like to keep at least a dozen versions available in case someone needs to go back to v1.3.2 for some reason.

[davidanthoff]

Ok, so that scenario is definitely not working at the moment: right now juliaup always installs Julia versions into the same location where JULIA_DEPOT_PATH is pointing to, so at the moment package install location and Julia versions install location are hard linked.

Here is one way we could change this: we could change the juliaup story to look through all the entries in what would be DEPOT_PATH, starting from the last entry, and as soon as it finds a depot with a juliaup config file in it, it will work off that depot. So if someone then had a juliaup config file in /usr/local/share/julia (which is already in the DEPOT_PATH, then that is where Julia versions would be stored and retrieved from, while packages would still be installed into ~/.julia. And if a user doesn't have write access to /usr/local/share/julia, then they couldn't install new Julia versions there.

Supporting that scenario will require a bunch of other things to be changed, probably, but maybe this could be a rough design?

[dmbates]

That seems reasonable. Thanks for considering this. For the time being we will go back to having several different julia-1.x.y links in our equivalent of usr/local/bin/

[StefanKarpinski]

Getting updates would be a good task for a cron job. The tricky bit would be making sure it's atomic in case someone tries to use a Julia version right when it's being updated.

[davidanthoff]

Getting updates would be a good task for a cron job. The tricky bit would be making sure it's atomic in case someone tries to use a Julia version right when it's being updated.

Both of that is implemented already :) The cron job is an advanced option alternative that one can pick at install time, and I generally use a global file lock to sync all concurrent operations. Now, that stuff is hard, so I wouldn't be surprised if there is a bug here or there, but in principle it should be all ok.

[KristofferC]

Just checking if there are any more thoughts here about a good way this could be implemented or alternatively if there is a good way to support this based on the current juliaup.

[csnazell]

I've a similar use case that's not multi-user. I'm experimenting with Fedora Silverblue at present and what I'd like to do is put julia in its own distrobox container (podman under the covers) & manage it via juliaup. The curl | sh approach, suitably sudo'd, to install into /opt/julia in the container works but the default Julia installation winds up in the user directory. I don't object to the packages going here but I want to put Julia + runtime into the container.

My current approach at the moment is using the tarball and manually managing my Julia installation with a custom JULIA_DEPOT_PATH to point to my home directory.

I'm just getting started with Julia so there's maybe something I missing.

I'm happy to bump this to a separate ticket if you prefer.