search

roualdes / bridgestan

BridgeStan provides efficient in-memory access through Python, Julia, and R to the methods of a Stan model.
https://roualdes.github.io/bridgestan
BSD 3-Clause "New" or "Revised" License
87 stars 12 forks source link

Consider moving Julia download away from LazyArtifacts #226

Closed miguelbiron closed 2 months ago

miguelbiron commented 2 months ago

Hi -- just ran into this weird bug where an error is raised on Julia 1.10 while downloading bridgestan artifact inside a github CI Windows runner

https://github.com/Julia-Tempering/Pigeons.jl/actions/runs/8636249839/job/23678766648#step:8:1159

Interestingly this doesn't happen on Julia 1.9

https://github.com/Julia-Tempering/Pigeons.jl/actions/runs/8636249839/job/23678765917#step:8:1136

WardBrian commented 2 months ago

Definitely weird!

Our CI wouldn't catch this because we only test that the download succeeds, not the permissions of the files, but it sounds like this is a Julia 1.10 regression: https://github.com/JuliaLang/julia/issues/52272

Maybe us putting something we want write access to into the artifacts directory is bad manners, I'm not sure. If so we could switch to what we do in Python etc and put it under ~/.bridgestan

@sethaxen originally added this feature for us so I'm going to ping him to see if he has any thoughts

miguelbiron commented 2 months ago

Ah! Nice find, it didn't occur to me that this would be affecting other packages... it definitely looks an important regression on Julia 1.10. Will keep digging to see if there's anything I can do on my end, thank you.

WardBrian commented 2 months ago

I got confirmation that we probably shouldn't be using the artifacts system for this: https://github.com/JuliaLang/julia/issues/52272#issuecomment-2050516371

They recommended https://github.com/JuliaPackaging/Scratch.jl, which seems similar, but I'm leaning toward using ~/.bridgestan like we do in Python or R. Thoughts @roualdes @sethaxen?

roualdes commented 2 months ago

I vote to just use ~/.bridgestan instead of Scratch.jl, lest we find later on some unknown unknowns about Scratch.jl and its personal philosophies. We theoretically know a lot about using ~/.bridgstan across various machines.

sethaxen commented 2 months ago

I don't have any particular insight here. Perhaps a benefit of Scratch would be to reduce the maintenance burden of this package by outsourcing management to that package (including, it seems, auto-deletion of the scratch space if the package is deleted or if it hasn't been used in a long time). But if the R and Python versions are already managing ~/.bridgestan, doing the same with Julia brings consistency and shouldn't greatly increase the maintenance burden.