search

JuliaPackaging / BinaryBuilder.jl

Binary Dependency Builder for Julia
https://binarybuilder.org
Other
391 stars 100 forks source link

Query string in download URL breaks wizard #877

Open helgee opened 4 years ago

helgee commented 4 years ago

Julia 1.5

julia> versioninfo()
Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.7.0)
  CPU: Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, icelake-client)
Environment:
  JULIA_PKG_DEVDIR = /Users/helge/projects/julia

Docker Desktop

grafik

BinaryBuilder

[12aac903] BinaryBuilder v0.2.5

Error Log

            # Step 3: Build for Linux(:x86_64, libc=:glibc)

You will now be dropped into the cross-compilation environment.
Please compile the package. Your initial compilation target is x86_64-linux-gnu
The $prefix environment variable contains the target directory.
Once you are done, exit by typing `exit` or `^D`

ArgumentError: '/var/folders/vp/qb910zys4w3951w9ntmb3zw40000gn/T/jl_2cTTVD/1HHxGCt5/srcdir' exists. `force=true` is required to remove '/var/folders/vp/qb910zys4w3951w9ntmb3zw40000gn/T/jl_2cTTVD/1HHxGCt5/srcdir' before copying.
Stacktrace:
 [1] checkfor_mv_cp_cptree(::String, ::String, ::String; force::Bool) at ./file.jl:308
 [2] cp(::String, ::String; force::Bool, follow_symlinks::Bool) at ./file.jl:345
 [3] cp at ./file.jl:345 [inlined]
 [4] setup(::BinaryBuilder.SetupSource{FileSource}, ::String, ::Bool) at /Users/helge/.julia/packages/BinaryBuilder/9oqch/src/Prefix.jl:311
 [5] setup_workspace(::String, ::Array{Any,1}; verbose::Bool) at /Users/helge/.julia/packages/BinaryBuilder/9oqch/src/Prefix.jl:368
 [6] step34(::BinaryBuilder.WizardState) at /Users/helge/.julia/packages/BinaryBuilder/9oqch/src/wizard/interactive_build.jl:510
 [7] run_wizard(::Nothing) at /Users/helge/.julia/packages/BinaryBuilder/9oqch/src/Wizard.jl:97
 [8] run_wizard() at /Users/helge/.julia/packages/BinaryBuilder/9oqch/src/Wizard.jl:80
 [9] top-level scope at REPL[1]:1
 [10] eval(::Module, ::Any) at ./boot.jl:331
 [11] eval_user_input(::Any, ::REPL.REPLBackend) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:134
 [12] repl_backend_loop(::REPL.REPLBackend) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:195
 [13] start_repl_backend(::REPL.REPLBackend, ::Any) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:180
 [14] run_repl(::REPL.AbstractREPL, ::Any; backend_on_current_task::Bool) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:292
 [15] run_repl(::REPL.AbstractREPL, ::Any) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:288
 [16] (::Base.var"#806#808"{Bool,Bool,Bool,Bool})(::Module) at ./client.jl:399
 [17] #invokelatest#1 at ./essentials.jl:710 [inlined]
 [18] invokelatest at ./essentials.jl:709 [inlined]
 [19] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:383
 [20] exec_options(::Base.JLOptions) at ./client.jl:313
 [21] _start() at ./client.jl:506

WizardState [step3]

I tried starting from scratch multiple times, rebooted the machine, and deleted the offending directories. Same result...

fingolfin commented 4 years ago

Perhaps you could add in some information, like: which versions of macOS, Julia, BinaryBuilder are you using? Is Docker installed (that's required for doing this on macOS)?

helgee commented 4 years ago

Perhaps you could add in some information, like: which versions of macOS, Julia, BinaryBuilder are you using? Is Docker installed (that's required for doing this on macOS)?

Indeed. I was just exhausted after a day of failure and needed to collect myself first. See above...

The issue only occurs when I try to use these sources from SourceForge:

https://downloads.sourceforge.net/project/libtirpc/libtirpc/1.2.6/libtirpc-1.2.6.tar.bz2
helgee commented 4 years ago

The issue is macOS-specific. No problems on Linux...

staticfloat commented 4 years ago

Hmmm, I have been unable to reproduce this on my MacOS machine, using the exact same versions of everything. Are you still facing this issue?

helgee commented 4 years ago

I just tried it again and it's gone 🤷 Thanks for having a look!

helgee commented 4 years ago

I have the same problem on a different machine now 😱 (using a different SourceForge link)

IIRC the only thing that the two setups have in common is that I installed Docker immediately before using BinaryBuilder. Could this be some kind of initialization thing?

fingolfin commented 4 years ago

These sourceforge links all use http redirects. That might be part of the problem.

helgee commented 4 years ago

The issue is back on the other Mac as well.

helgee commented 4 years ago

Okay, I did some digging.

The problem seems to be related to the query parameters that SourceForge attaches to the direct link which need to be manually removed.

The initial direct link from SF is, e.g.

https://downloads.sourceforge.net/project/gmat/GMAT/GMAT-R2020a/GMAT-src-R2020a.zip?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fgmat%2Ffiles%2FGMAT%2FGMAT-R2020a%2FGMAT-src-R2020a.zip%2Fdownload&ts=1597900593&use_mirror=autoselect

If I do not remove the query string garbage, the source becomes

source::BinaryBuilder.SetupSource{FileSource} = BinaryBuilder.SetupSource{FileSource}("/var/folders/vp/qb910zys4w3951w9ntmb3zw40000gn/T/jl_7axuQ3/GMAT-src-R2020a.zip?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fgmat%2Ffiles%2FGMAT%2FGMAT-R2020a%2FGMAT-src-R2020a.zip%2Fdownload&ts=1597900593&use_mirror=autoselect", "943f403ac04d958b313b1d99d64fd09e3fa8e4c65363809d5bb88dd8c66e43e4", "")

The file var/folders/vp/qb910zys4w3951w9ntmb3zw40000gn/T/jl_7axuQ3/GMAT-src-R2020a.zip?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fgmat%2Ffiles%2FGMAT%2FGMAT-R2020a%2FGMAT-src-R2020a.zip%2Fdownload&ts=1597900593&use_mirror=autoselect exists on disk and if I try to cp it, e.g. cp(garbage_name_above, "Desktop"), I get the same error message:

julia> blob = "/var/folders/vp/qb910zys4w3951w9ntmb3zw40000gn/T/jl_7axuQ3/GMAT-src-R2020a.zip?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fgmat%2Ffiles%2FGMAT%2FGMAT-R2020a%2FGMAT-src-R2020a.zip%2Fdownload&ts=1597900593&use_mirror=autoselect"
"/var/folders/vp/qb910zys4w3951w9ntmb3zw40000gn/T/jl_7axuQ3/GMAT-src-R2020a.zip?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fgmat%2Ffiles%2FGMAT%2FGMAT-R2020a%2FGMAT-src-R2020a.zip%2Fdownload&ts=1597900593&use_mirror=autoselect"

shell> pwd
/Users/helge

julia> cp(blob, "Desktop")
ERROR: ArgumentError: 'Desktop' exists. `force=true` is required to remove 'Desktop' before copying.
Stacktrace:
 [1] checkfor_mv_cp_cptree(::String, ::String, ::String; force::Bool) at ./file.jl:308
 [2] cp(::String, ::String; force::Bool, follow_symlinks::Bool) at ./file.jl:345
 [3] cp(::String, ::String) at ./file.jl:345
 [4] top-level scope at REPL[3]:1

EDIT: The error is correct because Desktop does in fact exist. My bad...

TL;DR: There seem to be two issues here: - A bug in Base.cp which gives an incorrect error message for a garbage source argument

What is still puzzling me is that I ran into a situation were the downloaded archive became GMAT-src-R2020a.zip? but if we apply Occam's Razor the answer is probably that I had a trailing ? because I edited the URL improperly.

bramtayl commented 2 years ago

I ran into this and the problem was the url had ".ZIP" instead of ".zip" at the end.