search

JuliaEcosystem / PkgDeps.jl

More insights about your packages dependencies
MIT License
23 stars 6 forks source link

Support registries in gzipped tarballs #38

Closed danielmatz closed 1 year ago

danielmatz commented 2 years ago

Starting with Julia 1.7, a registry can also be stored in a depot as a gzipped tarball. This commit unpacks the tarball into a temporary directory. This was the simplest way to support this new feature.

This implementation will unpack a fresh copy of the registry on every call to reachable_registries. We could instead cache the temporary directory for future calls.

There's an open PR for the Tar.jl package that will be of future interest:

https://github.com/JuliaIO/Tar.jl/pull/95

This would allow us to instead stream the individual files straight out of the tarball, rather than having to unpack it into a temporary location.

danielmatz commented 2 years ago

I don't think this is a good approach. Pkg has its own API to analyse registries, whether they are gzipped or not, should use that instead

Yes, I see Pkg has a Pkg.Registry.reachable_registries function as of Julia 1.7.

Are you proposing that this package require Julia 1.7 or newer and remove its duplicate implementations of the registry-related code?

Or should I detect the Julia version and use the Pkg API if available, and otherwise fall back to the existing registry code in PkgDeps, which would not support tarballs?

giordano commented 2 years ago

I guess that's a question for @mattBrzezinski. The question boils down to whether supporting julia v1.6 is important, or we can drop it and move to v1.7+ only. We have a similar situation in PackageAnalyzer, there I'd be ok with dropping v1.6 and moving to v1.7+ only.

danielmatz commented 2 years ago

OK, I made a new PR with the approach of requiring Julia 1.7. See #39.

mattBrzezinski commented 2 years ago

I guess that's a question for @mattBrzezinski. The question boils down to whether supporting julia v1.6 is important, or we can drop it and move to v1.7+ only. We have a similar situation in PackageAnalyzer, there I'd be ok with dropping v1.6 and moving to v1.7+ only.

I would prefer to maintain support for LTS versions of Julia.

danielmatz commented 2 years ago

Well, the LTS version of Julia won't create gzipped tarballs. I bumped the julia compat entry in my other PR. So, I suppose we could release v0.7 of this package, which would require Julia 1.7, while maintaining v0.6 for the LTS version of Julia?

Alternatively, I could rework my PR so that it uses the Pkg API for Julia 1.7+, but gracefully falls back to your existing code for the LTS.

mattBrzezinski commented 2 years ago

We could instead do something nifty like this to handle the differences between versions!

danielmatz commented 2 years ago

OK, I took a stab at it in my other PR.

danielmatz commented 1 year ago

Closing this in favor of #42.