Closed lmiq closed 2 years ago
Can confirm and it was discussed in #6
julia> update_julia("1.7")
installing the latest version of julia (1.7.0) from https://julialang-s3.julialang.org/bin/linux/x64/1.7/julia-1.7.0-linux-x86_64.tar.gz
ERROR: IOError: mkdir("/opt/julias"; mode=0o777): permission denied (EACCES)
Stacktrace:
[1] uv_error
@ ./libuv.jl:97 [inlined]
[2] mkdir(path::String; mode::UInt16)
@ Base.Filesystem ./file.jl:183
[3] mkpath(path::String; mode::UInt16)
@ Base.Filesystem ./file.jl:234
[4] mkpath
@ ./file.jl:229 [inlined]
[5] extract
@ ~/.julia/packages/UpdateJulia/ANLSe/src/UpdateJulia.jl:204 [inlined]
[6] (::UpdateJulia.var"#3#5"{String, VersionNumber})(file::String)
@ UpdateJulia ~/.julia/packages/UpdateJulia/ANLSe/src/UpdateJulia.jl:66
[7] download_delete(f::UpdateJulia.var"#3#5"{String, VersionNumber}, url::String)
@ UpdateJulia ~/.julia/packages/UpdateJulia/ANLSe/src/UpdateJulia.jl:176
[8] update_julia(version::String; set_default::Bool, install_location::String, bin::String, os_str::String, arch::String, prefer_gui::Bool)
@ UpdateJulia ~/.julia/packages/UpdateJulia/ANLSe/src/UpdateJulia.jl:65
[9] update_julia(version::String)
@ UpdateJulia ~/.julia/packages/UpdateJulia/ANLSe/src/UpdateJulia.jl:45
[10] top-level scope
@ REPL[4]:1
Expecting root access by default is not the intended behavior, and I agree that it is not good. I'm investigating.
On the Ubuntu CI testing machine, /opt
exists and has all permissions: dwrewrewre
. I imagine one of these is not the case on your machines. I have UpdateJulia use /opt/julias
as a location to store the full Julia executables in an attempt to follow FHS. I'm not a Linux sysadmin expert, though. Where should one keep these executables? Where do your Julia installations live?
then I tried to set install and bin dir to my custom directories, but the error persisted.
@lmiq The install_location
argument was not properly handled for Linux. That should now be fixed in main & 0.1.1, so you should be able to set install and bin locations to your custom directories and have it work without root.
Still need to find a better default install location, though.
I currently install everything on /home/user/programs/julia
, I would suggest that a good default option is just /home/user/julia
. I don't see why there is the need for installation_location
and another binary
location.
Within the juila/
folder, I have the installation of different versions, i. e.
/home/user/julia/julia-1.6.3
/home/user/julia/julia-1.7.0
and a symlink for the latest binary as
julia -> ./julia-1.7.0/bin/julia*
Just changing that symlink makes everything (vscode etc) to work with the latest version.
A lot of people (including me) don't like if the home directory is altered by programs.
I believe, folders like /home/user/.local/bin
are better suited.
See for example here: https://unix.stackexchange.com/a/651928
Yes, that would be fine. But no need for one directory for installation and other for binaries, IMO. Anyway, the original "error" is solved, so I'm closing the issue and let the developer decide what is the best default behavior. Thanks.
I don't think it's resolved?
(Edit: I did see that you called the update differently. But still, my error message persists and I guess almost all Linux user would run into this issue!)
Right now (on Arch Linux) I did:
(@v1.7) pkg> activate --temp
Activating new project at `/tmp/jl_0Bwgo6`
(jl_0Bwgo6) pkg> add https://github.com/LilithHafner/UpdateJulia.jl
Cloning git-repo `https://github.com/LilithHafner/UpdateJulia.jl`
Updating git-repo `https://github.com/LilithHafner/UpdateJulia.jl`
Updating registry at `~/.julia/registries/General`
Updating git-repo `https://github.com/JuliaRegistries/General.git`
Resolving package versions...
Updating `/tmp/jl_0Bwgo6/Project.toml`
[770da0de] + UpdateJulia v0.1.1 `https://github.com/LilithHafner/UpdateJulia.jl#main`
Updating `/tmp/jl_0Bwgo6/Manifest.toml`
[682c06a0] + JSON v0.21.2
[69de0a69] + Parsers v2.1.2
[fd094767] + Suppressor v0.2.0
[770da0de] + UpdateJulia v0.1.1 `https://github.com/LilithHafner/UpdateJulia.jl#main`
[ade2ca70] + Dates
[a63ad114] + Mmap
[de0858da] + Printf
[4ec0a83e] + Unicode
Precompiling project...
1 dependency successfully precompiled in 1 seconds (3 already precompiled)
julia> using UpdateJulia
julia> update_julia("1.3")
installing julia 1.3.1 from https://julialang-s3.julialang.org/bin/linux/x64/1.3/julia-1.3.1-linux-x86_64.tar.gz
This version is out of date. The latest official release is 1.7.0
ERROR: IOError: mkdir("/opt/julias"; mode=0o777): permission denied (EACCES)
Stacktrace:
[1] uv_error
@ ./libuv.jl:97 [inlined]
[2] mkdir(path::String; mode::UInt16)
@ Base.Filesystem ./file.jl:183
[3] mkpath(path::String; mode::UInt16)
@ Base.Filesystem ./file.jl:234
[4] mkpath
@ ./file.jl:229 [inlined]
[5] extract
@ ~/.julia/packages/UpdateJulia/p49mp/src/UpdateJulia.jl:204 [inlined]
[6] (::UpdateJulia.var"#3#5"{String, VersionNumber})(file::String)
@ UpdateJulia ~/.julia/packages/UpdateJulia/p49mp/src/UpdateJulia.jl:66
[7] download_delete(f::UpdateJulia.var"#3#5"{String, VersionNumber}, url::String)
@ UpdateJulia ~/.julia/packages/UpdateJulia/p49mp/src/UpdateJulia.jl:176
[8] update_julia(version::String; set_default::Bool, install_location::String, bin::String, os_str::String, arch::String, prefer_gui::Bool)
@ UpdateJulia ~/.julia/packages/UpdateJulia/p49mp/src/UpdateJulia.jl:65
[9] update_julia(version::String)
@ UpdateJulia ~/.julia/packages/UpdateJulia/p49mp/src/UpdateJulia.jl:45
[10] top-level scope
@ REPL[4]:1
I really think the installation path should be changed. On a Ubuntu machine the following path already exists. So python seems to install some stuff there:
fxw@FA8-24158:~$ ls .local/bin/
f2py f2py3 f2py3.8 imageio_download_bin imageio_remove_bin lsm2bin napari pint-convert skivi tiff2fsspec tiffcomment tifffile tqdm
On the same Ubuntu machine (where I don't have root permissions) it errors as well
(@v1.6) pkg> activate --temp
Activating new environment at `/tmp/jl_kKClqj/Project.toml`
(jl_kKClqj) pkg> add (@v1.7) pkg> activate --temp
Activating new project at `/tmp/jl_0Bwgo6`
(jl_0Bwgo6) pkg> add https://github.com/LilithHafner/UpdateJulia.jl
Cloning git-repo `https://github.com/LilithHafner/UpdateJulia.jl`
Updating git-repo `https://github.com/LilithHafner/UpdateJulia.jl`
Updating registry at `~/.julia/registries/General`
Updating git-repo `https://github.com/JuliaRegistries/General.git`
Resolving package versions...
Updating `/tmp/jl_0Bwgo6/Project.toml`
[770da0de] + UpdateJulia v0.1.1 `https://github.com/LilithHafner/UpdateJulia.jl#main`
Updating `/tmp/jl_0Bwgo6/Manifest.toml`
(jl_kKClqj) pkg> add (@v1.7) pkg> activate --temp
Activating new project at `/tmp/jl_0Bwgo6`
(jl_0Bwgo6) pkg> add https://github.com/LilithHafner/UpdateJulia.jl
Cloning git-repo `https://github.com/LilithHafner/UpdateJulia.jl`
Updating git-repo `https://github.com/LilithHafner/UpdateJulia.jl`
Updating registry at `~/.julia/registries/General`
Updating git-repo `https://github.com/JuliaRegistries/General.git`
Resolving package versions...
Updating `/tmp/jl_0Bwgo6/Project.toml`
[770da0de] + UpdateJulia v0.1.1 `https://github.com/LilithHafner/UpdateJulia.jl#main`
Updating `/tmp/jl_0Bwgo6/Manifest.toml`
[682c06a0] + JSON v0.21.2
[69de0a69] + Parsers v2.1.2
(jl_kKClqj) pkg> add https://github.com/LilithHafner/UpdateJulia.jl
Cloning git-repo `https://github.com/LilithHafner/UpdateJulia.jl`
Updating git-repo `https://github.com/LilithHafner/UpdateJulia.jl`
Updating registry at `~/.julia/registries/General`
Resolving package versions...
Installed Suppressor ─ v0.2.0
Updating `/tmp/jl_kKClqj/Project.toml`
[770da0de] + UpdateJulia v0.1.1 `https://github.com/LilithHafner/UpdateJulia.jl#main`
Updating `/tmp/jl_kKClqj/Manifest.toml`
[682c06a0] + JSON v0.21.2
[69de0a69] + Parsers v2.1.2
[fd094767] + Suppressor v0.2.0
[770da0de] + UpdateJulia v0.1.1 `https://github.com/LilithHafner/UpdateJulia.jl#main`
[ade2ca70] + Dates
[a63ad114] + Mmap
[de0858da] + Printf
[4ec0a83e] + Unicode
Precompiling project...
2 dependencies successfully precompiled in 2 seconds (2 already precompiled)
julia> using UpdateJulia
julia> update_julia("1.7")
installing the latest version of julia (1.7.0) from https://julialang-s3.julialang.org/bin/linux/x64/1.7/julia-1.7.0-linux-x86_64.tar.gz
ERROR: IOError: mkdir("/opt/julias"; mode=0o777): permission denied (EACCES)
Stacktrace:
[1] uv_error
@ ./libuv.jl:97 [inlined]
[2] mkdir(path::String; mode::UInt16)
@ Base.Filesystem ./file.jl:179
[3] mkpath(path::String; mode::UInt16)
@ Base.Filesystem ./file.jl:230
[4] mkpath
@ ./file.jl:225 [inlined]
[5] extract
@ ~/.julia/packages/UpdateJulia/p49mp/src/UpdateJulia.jl:204 [inlined]
[6] (::UpdateJulia.var"#3#5"{String, VersionNumber})(file::String)
@ UpdateJulia ~/.julia/packages/UpdateJulia/p49mp/src/UpdateJulia.jl:66
[7] download_delete(f::UpdateJulia.var"#3#5"{String, VersionNumber}, url::String)
@ UpdateJulia ~/.julia/packages/UpdateJulia/p49mp/src/UpdateJulia.jl:176
[8] update_julia(version::String; set_default::Bool, install_location::String, bin::String, os_str::String, arch::String, prefer_gui::Bool)
@ UpdateJulia ~/.julia/packages/UpdateJulia/p49mp/src/UpdateJulia.jl:65
[9] update_julia(version::String)
@ UpdateJulia ~/.julia/packages/UpdateJulia/p49mp/src/UpdateJulia.jl:45
[10] top-level scope
@ REPL[4]:1
Thank you both for your comments! I agree that the default install location (and presumably bin location) should be changed, but I'm not sure what the right directories are.
Paths that came up in this thread are:
/home/user/.local/bin
/home/user/programs/julia
/home/user/julia
It seems to me that there really ought to be a canonical install location so that users can expect to find Python, Julia, and whatever else they personally installed in the same place (on mac I think it's /Applications
).
A lot of people (including me) don't like if the home directory is altered by programs.
Is this just /home
, or also subdirectories?
I consider everything inside of /home/user
as bad.
Unless it is /home/user/.whatever
, so a hidden directory with a leading dot.
I mean, Pluto.jl for example stores temporary notebooks inside /home/fxw/.julia/pluto_notebooks
.
Makie.jl as well:
╭─fxw@earth ~/.julia/makiegallery
╰─➤ ls [2-12-21 | 9:59:47]
╭─fxw@earth ~/.julia/makiegallery
╰─➤ pwd [2-12-21 | 9:59:48]
/home/fxw/.julia/makiegallery
If they do it, why this package not?
Maybe something like: /home/USER/.julia/UpdateJulia/bin/
is good?
Then it stays inside the .julia
related folder and is not scattered around some random location.
Oooh, I like keeping things in .julia! Makes uninstallation easier.
I believe, I read somewhere from Maintainers that this was not intended and could cause problems in the future if they want to use that name.
But UpdateJulia
is probably a folder name they would not choose?
Can't find the comment though.
The artifacts system might be applicable. (EDIT: seems like artifacts are for downloading, not local storage)
Could work as well!
But the artifacts are downloaded and stored, right? They won't be deleted since the packages rely on them during runtime. Similarly, you also download the Julia binary as artifact.
But I'm not sure
I'll look into artifacts more for the installation, but for the bin, we should ideally use something that, according to UNIX or at least Linux standards is already on the path.
echo $PATH
On my Ubuntu machine:
/home/fxw/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
However, Arch Linux has not any path which has writing rights for the fxw
user.
/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
But Arch Linux is always cleaner than Ubuntu.
Thanks!
I imagine the only path entry that fxw
has writing writes to on the first list is /home/fwx/.local/bin
?
For Arch Linux, if path itself is not user-writable, and neither are its contents, then is it possible for a user to add software to their user path at all?
echo $PATH
on my mac:
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
Arch Linux users are quite familiar with changing PATH etc. Maybe that originates from there. It would be OK for me to change my PATH. It just needs to be documented well
On Mac /usr/local/bin
can be written as user?
julia> stat("/usr/local/bin")
StatStruct for "/usr/local/bin"
size: 10304 bytes
device: 16777220
inode: 942938
mode: 0o040775 (drwxrwxr-x)
nlink: 322
uid: 501 (x)
gid: 80 (admin)
rdev: 0
blksz: 4096
blocks: 0
mtime: 2021-12-02T00:32:46-0600 (8 hours ago)
ctime: 2021-12-02T00:32:46-0600 (8 hours ago)
Seems not, but it works fine on my machine to edit it (e.g. touch /usr/local/bin
, and update_julia()
). Prolly because my default user is admin on my mac, which, I believe, is normal for personal ocmputers.
Putting the julia instalation inside an UpdateJulia
directory (anywhere) is strange to me, gives too much importance to the package. I would with /home/user/.local/julia
by default.
There are some other update tools around there, I think somewhat commented on a pip one. Ideally these tools should do the same, so one can use or another at will.
I think mac is in the same situation as Arch: nothing on the default path is non-admin-user writable, so it is necessary to have an administrator either add something to path (unlikely) or add something to a bin. That does need to be documented well, and not involve mysterious errors.
For ubuntu bin, I lean toward $(homedir())/.local/bin
if it occursin
ENV["PATH"]
, and otherwise telling the user that they will have to link manually.
For mac bin, I lean toward keeping /usr/local/bin
, and if the user can't write to that folder, then skipping linking, telling them to run as administrator to link (or link manually).
In either failure case, we also print the fully qualified executable location for users who can't or won't put it on their path and want to e.g. paste that location into the juno julia executable configuration field.
$(homedir())/.local/julia
seems like a reasonable installation location, but I'd really like to use a canonical location that all or almost all user-installed software is placed in. It seems this sort of recommendation should be in the Unix or Linux specs somewhere, but I have yet to find an authoritative source.
If there would be a canonical solution, someone would have mentioned it here: https://unix.stackexchange.com/a/651928 (Maybe read the full conversation)
But a few comments suggest what we agreed on. Looks good to me!
UpdateJulia refers to two (possibly) different paths: bin and install location.
(On mac I have the Julia executable at /Applications/Julia-1.7.0.app/Contents/Resources/julia/bin/julia
. The install location is /Applications
and the bin is /user/local/bin
. /user/local/bin/julia-1.7
is linked to /Applications/Julia-1.7.0.app/Contents/Resources/julia/bin/julia
.)
@roflmaostc I believe we agreed on $(homedir())/.local/bin
as a bin location, and the thread you linked seems to support that. What about install location, or is the julia installation on UNIX a single executable file?
The instruction in linux is just to unpack the .tar.gz
, which creates a directory structure:
/julia-1.7.0% ls
bin/ etc/ include/ lib/ libexec/ LICENSE.md share/
inside bin
there is the julia executable. I don't know if it can be copied somewhere else. I have been using a symbolic link.
I imagine it is not a good idea to move the executable elsewhere and discard the rest. For the reason, unless we're adding $install_location/julia-1.7.0/bin/
to path, we need to use a symbolic link. If we let install_location = bin
, then we have a name conflict between the symlink at $bin/julia-1.7.0
and the directory at install_location/julia-1.7.0
, which is why I'm still looking for a canonical install_location. In other words, where should we unzip the .tar.gz file to?
@lmiq uses /home/user/programs/julia
@roflmaostc tossed out $(homedir())/.julia/...
as an option
(side note: on mac I use /Applications
)
perhaps $(homedir())/.local/programs/julia
, $(homedir())/.local/julia
, or $(homedir())/.local/programs
?
$(homedir())/.local/bin
sounds reasonable for the link of the binary!
Install location I don't know 😆
Is it possible to get it running with artifacts?
Downloading the Julia .tar.gz
and unpacking it inside the artifacts folder sounds ok?
Then we can sym link bin/julia
to $(homedir())/.local/bin
?
Another issue:
Such that a new call of julia
would point to the new version, you would have to overwrite the old symlink (which probably was existing before the user was using UpdateJulia.jl, wouldn't you?
One drawback of artifacts (or .julia) for install location is that that should be a platform-independent solution, but on mac pretty much all software lives in /Applications
, so it would likely be confusing to users not to find it there. That is, I prefer to follow the operating system conventions rather than the (non-existent?) Julia conventions on where to store the Julia instillation.
Yeah, I'm not sure if such a general folder exists in Linux? Most users install software with root rights so it's not in the userspace located?
Yes. We overwrite the old symlink whenever the set_default
keyword argument is true in update_julia(..., set_default=...)
. By default, set_default
is true
whenever the version to update to is unspecified (e.g. update_julia()
will install the latest version and set it as default, but
update_julia("1.3")will install 1.3.x and not set it as the default). Perhaps
set_defaultshould also be true by default for
update_julia("1.7"),
update_julia("1.7.0"), etc.
Most users install software with root rights
Do folks run installation tools with root rights or do everything by hand?
Some via package manager, some via hand, some use jill.
Btw, I also asked in Slack, see here: https://julialang.slack.com/archives/C6A044SQH/p1638462931034400?thread_ts=1638456258.026400&cid=C6A044SQH
jill uses opt/julias
with root and $HOME/packages/julias
without for install location and /usr/local/bin
and $HOME/.local/bin
for bin location (they also use the terms download and install rather than install and bin, respectively.)
All those seem reasonable, though $HOME/packages/julias
is not a hidden directory.
My tentative plan for default locations with/without root is in the readme.md of https://github.com/LilithHafner/UpdateJulia.jl/tree/Document-installation-locations
OS | System install | System bin | User install | User bin |
---|---|---|---|---|
unix | /opt/julias |
/usr/local/bin |
~/.local/julias |
~/.local/bin |
mac | /Applications |
/usr/local/bin |
~/Applications |
~/.local/bin |
windows | \Program Files |
automatically add install location to path | ~\AppData\Local\Programs |
automatically add install location to path |
Personally I would prefer if UpdateJulia
is included in the path name?
Otherwise, it might be hard to find the files (if someones searches for them?)
Putting the julia instalation inside an UpdateJulia directory (anywhere) is strange to me, gives too much importance to the package
Personally I would prefer if UpdateJulia is included in the path name?
Obviously, we can't do both...
Points against UpdateJulia in the path
Points for UpdateJulia in the path
UpdateJulia and jill can interop (as far as I'm aware) whether or not they share a download location. Each can install whatever version wherever and link to it from any bin on the path.
I lean toward putting the install location into the readme (it's already in the docstring for update_julia()) and not putting UpdateJulia in the path. If folks know that their Julia was installed via UpdateJulia, they can read the documentation, and if they don't they can poke around in their system and (if we choose a good default location) find it pretty easily.
FWIW, the jill developer also doesn't know of a definitive download location off-hand https://github.com/abelsiqueira/jill/issues/61#issuecomment-984887660
Wouldn't it be a possibility to derive the installation directory from the current julia installation's path?
Everyone has already made his/her decision on where to install julia.
Or you could accept :auto
for installation in the home folder of the current installation.
EDIT: I thought of something like
install_location == :auto && ( install_location = dirname(dirname(first(Base.julia_cmd()))) )
@hhaensel, I really like using the currently installed Julia location! I think we need 3 dirname
s to get both installations in parallel.
I'm thinking 1) Decide whether to install for all users or only current user:
startswith(first(Base.julia_cmd()), homedir())
, install for userdirname(dirname(Base.Sys.BINDIR))
but if that location is /
, or we are installing at a different level than the existing installation (at user request possibly due to lack of privileges) then use the default locations (/opt
and ~/.local
for Linux).
number of dirname
s is system dependant. On mac it is 5, on Linux & windows it is 2. (EDIT due to @hhaensel)/opt
, /opt/julia
, or /opt/julias
. On Mac and windows it is exactly the default install location.OS | System install | System bin | User install | User bin |
---|---|---|---|---|
unix | /opt |
/usr/local/bin |
~/.local * |
~/.local/bin |
mac | /Applications |
/usr/local/bin |
~/Applications |
~/.local/bin |
windows | \Program Files |
automatically add install location to path | ~\AppData\Local\Programs |
automatically add install location to path |
*prefer to use the install location of currently running Julia where possible
That sounds good.
Just realised that somehow Base.julia_cmd()
produces strange results, if called from a link, at least for Windows.
C:\temp>julialink.exe -Jc:\Julia\Julia-1.7.0\lib\julia\sys.dll
c:\Julia\Julia-1.7.0\lib\julia\sys.dll
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.7.0 (2021-11-30)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
julia> Base.julia_cmd()
`'C:\temp\julia.exe' -Cnative '-Jc:\Julia\Julia-1.7.0\lib\julia\sys.dll' -g1`
It seems that Base.julia_cmd()
is identical to
joinpath(Base.Sys.BINDIR, Base.julia_exename())
on Windows.
Just tested on a wsl (Ubuntu on windows): No problem, the link is resolved to its origin before executing and Base.Sys.BINDIR
is the target directory and not the link directory. I think, no one will use links on a Windows system, so we're safe with the proposed solution.
help?> Base.Sys.BINDIR
Sys.BINDIR
A string containing the full path to the directory containing the julia
executable.
This is what we're looking for. If it doesn't behave as the docstring indicates on windows, that's a bug in Base.Sys.BINDIR
that should prolly be reported.
yip, I agree
@hhaensel @roflmaostc @lmiq, #12 is ready for review if any of you would be willing to look it over. The parts directly relevant to this thread are relatively short (lines 21–44 & 70–73) and noted in the comments of the PR. Other elements of the PR are testing and robustness improvements due to edge cases revealed in the implementation of this bin+install location redesign.
I got this error, then I tried to set install and bin dir to my custom directories, but the error persisted. IMHO expecting root access by default is not a good idea.