JuliaPackaging / BinaryBuilderBase.jl

https://juliapackaging.github.io/BinaryBuilderBase.jl/stable
MIT License
11 stars 31 forks source link

Create path for cached_git_clone if parent directory doesn't exist #317

Closed cserteGT3 closed 8 months ago

cserteGT3 commented 1 year ago

I was testing the resurrected MuJoCo jll build (PR here: https://github.com/JuliaPackaging/Yggdrasil/pull/7119), and errors like this ocurred:

C:\Users\myusername\Documents\GIT-work\Yggdrasil\M\MuJoCo>julia --project build_tarballs.jl --verbose
[ Info: Building for x86_64-w64-mingw32
    Updating registry at `C:\Users\myusername\.julia\registries\General.toml`
┌ Info: Using cached git repository
│   url = "https://github.com/deepmind/mujoco"
└   repo_path = "C:\\Users\\myusername\\.julia\\dev\\BinaryBuilderBase\\deps\\downloads\\clones\\mujoco-84d18077fafc7d59451b5f4cff1cc6f00032f8436725071864ae41c92685c118"
ERROR: LoadError: GitError(Code:ENOTFOUND, Class:Repository, could not find repository from 'C:\Users\myusername\.julia\dev\BinaryBuilderBase\deps\downloads\clones\mujoco-84d18077fafc7d59451b5f4cff1cc6f00032f8436725071864ae41c92685c118')
Stacktrace:
  [1] macro expansion
    @ C:\Users\myusername\.julia\juliaup\julia-1.9.2+0.x64.w64.mingw32\share\julia\stdlib\v1.9\LibGit2\src\error.jl:111 [inlined]
  [2] LibGit2.GitRepo(path::String)
    @ LibGit2 C:\Users\myusername\.julia\juliaup\julia-1.9.2+0.x64.w64.mingw32\share\julia\stdlib\v1.9\LibGit2\src\repository.jl:11
  [3] cached_git_clone(url::String; hash_to_check::String, downloads_dir::String, verbose::Bool, progressbar::Bool)
    @ BinaryBuilderBase C:\Users\myusername\.julia\dev\BinaryBuilderBase\src\Sources.jl:214
  [4] cached_git_clone
    @ C:\Users\myusername\.julia\dev\BinaryBuilderBase\src\Sources.jl:202 [inlined]
  [5] #download_source#39
    @ C:\Users\myusername\.julia\dev\BinaryBuilderBase\src\Sources.jl:252 [inlined]
  [6] download_source
    @ C:\Users\myusername\.julia\dev\BinaryBuilderBase\src\Sources.jl:251 [inlined]
  [7] #43
    @ .\broadcast.jl:1297 [inlined]
  [8] _broadcast_getindex_evalf
    @ .\broadcast.jl:683 [inlined]
  [9] _broadcast_getindex
    @ .\broadcast.jl:656 [inlined]
 [10] getindex
    @ .\broadcast.jl:610 [inlined]
 [11] macro expansion
    @ .\broadcast.jl:974 [inlined]
 [12] macro expansion
    @ .\simdloop.jl:77 [inlined]
 [13] copyto!
    @ .\broadcast.jl:973 [inlined]
 [14] copyto!
    @ .\broadcast.jl:926 [inlined]
 [15] copy
    @ .\broadcast.jl:898 [inlined]
 [16] materialize(bc::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1}, Nothing, Base.Broadcast.var"#43#44"{Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:verbose,), Tuple{Bool}}}, typeof(BinaryBuilderBase.download_source)}, Tuple{Vector{GitSource}}})
    @ Base.Broadcast .\broadcast.jl:873
 [17] autobuild(dir::AbstractString, src_name::AbstractString, src_version::VersionNumber, sources::Vector{<:BinaryBuilderBase.AbstractSource}, script::AbstractString, platforms::Vector, products::Vector{<:Product}, dependencies::Vector{<:BinaryBuilderBase.AbstractDependency}; verbose::Bool, debug::Bool, skip_audit::Bool, ignore_audit_errors::Bool, autofix::Bool, code_dir::Union{Nothing, String}, require_license::Bool, dont_dlopen::Bool, kwargs::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
    @ BinaryBuilder C:\Users\myusername\.julia\dev\BinaryBuilder\src\AutoBuild.jl:801
 [18] build_tarballs(ARGS::Any, src_name::Any, src_version::Any, sources::Any, script::Any, platforms::Any, products::Any, dependencies::Any; julia_compat::String, kwargs::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
    @ BinaryBuilder C:\Users\myusername\.julia\dev\BinaryBuilder\src\AutoBuild.jl:363
 [19] top-level scope
    @ C:\Users\myusername\Documents\GIT-work\Yggdrasil\M\MuJoCo\build_tarballs.jl:55
in expression starting at C:\Users\myusername\Documents\GIT-work\Yggdrasil\M\MuJoCo\build_tarballs.jl:55

I think the problem is, that ther folder where git wants to check out doesn't exist, therefore the cloning fails. This PR fixes is, by checking if the parent directory of the repo_path exists and creating it if not.

Let me know what you think about this

giordano commented 8 months ago

Sorry for the late review!

I think the problem is, that ther folder where git wants to check out doesn't exist

I don't think this is an accurate analysis. The output you shared shows the message Using cached git repository which comes from https://github.com/JuliaPackaging/BinaryBuilderBase.jl/blob/6faea2ef758b089156f35716d0f55dc74a8ab786/src/Sources.jl#L212 which is in a different if branch than what you're editing in this PR, so I don't see how this PR would fix that issue.

Side note, I'm even surprised that BinaryBuilder works at all on Windows, to the best of my knowledge it shouldn't work, see for example https://github.com/JuliaPackaging/BinaryBuilder.jl/issues/25 and https://github.com/JuliaPackaging/BinaryBuilder.jl/issues/520

cserteGT3 commented 8 months ago

If I remember correctly, the PR fixed my issue, but it was long time ago, and I abandoned the project since then. I suggest closing this PR.

Thank you for taking your time to review it!