Proposal: Allow redirects to repos

christhekeele commented 4 months ago

Usecase

I have a self-hosted repository in Fly Tigris object storage like: https://fly.storage.tigris.dev/my-hex-registry
I have a 302 redirect set up to it like https://hex.example.com to hide the storage implementation details
I would like mix hex.repo add my-hex https://hex.example.com to work instead of requiring the precise bucket location

Current Behaviour

Today I just get an error after about 30 seconds of what I assume is retrying in Mix.install:

Error

``` ** (Mix.Error) No package with name my-package (from: mix.exs) in registry (mix 1.15.7) lib/mix.ex:577: Mix.raise/2 (hex 2.0.6) lib/hex/remote_converger.ex:317: Hex.RemoteConverger.verify_package_req/4 (elixir 1.15.7) lib/enum.ex:984: Enum."-each/2-lists^foreach/1-0-"/2 (hex 2.0.6) lib/hex/remote_converger.ex:306: Hex.RemoteConverger.verify_input/2 (hex 2.0.6) lib/hex/remote_converger.ex:52: Hex.RemoteConverger.converge/2 (mix 1.15.7) lib/mix/dep/converger.ex:133: Mix.Dep.Converger.all/4 (mix 1.15.7) lib/mix/dep/converger.ex:89: Mix.Dep.Converger.converge/4 #cell:setup:13: (file) ```

Providing the full storage location works fine.

I believe this could be fixed in Hex.HTTP by giving :httpc the autoredirect: true option; it is given false today.

Proposal

I assume it is enforced off today for security and trust reasons I am not fluent in. I think it would be nice to make this opt-in configurable, unless doing so has implications lost on me that would compromise hex's general trust model.

I would propose parameterizing it, defaulting to false, via env vars similar to HEX_UNSAFE_REGISTRY=1. Perhaps something like HEX_FOLLOW_REDIRECTS=1?

wojtekmach commented 4 months ago

I think autoredirect: true by default would be fine given all repository resources are signed. Otherwise, adding an option to enable is fine by me too. Let's wait for some more feedback but I believe a PR would be welcome!

ericmj commented 4 months ago

We should be handling redirects: https://github.com/hexpm/hex/blob/0a5c49f59fe767a5dbeb56872408c8f015e575b7/lib/hex/http.ex#L113 Is it not working?

christhekeele commented 4 months ago

I'm still seeing the error reported above; but perhaps it is not in fact related to redirects. I will try to set up a repro this week.

ericmj commented 3 months ago

@christhekeele Did you learn anything more about this?

christhekeele commented 3 months ago

I started to try to reproduce the error above with DNS on a domain I control personally, but then got distracted by the chore of waiting for DNS to propagate from my fairly slow provider 😅 I will get back on the horse this weekend and sort out the repro

christhekeele commented 3 months ago

Yup, for some reason I just cannot get this error to raise:

** (Mix.Error) No package with name my-package (from: mix.exs) in registry
    (mix 1.15.7) lib/mix.ex:577: Mix.raise/2
    (hex 2.0.6) lib/hex/remote_converger.ex:317: Hex.RemoteConverger.verify_package_req/4
    (elixir 1.15.7) lib/enum.ex:984: Enum."-each/2-lists^foreach/1-0-"/2
    (hex 2.0.6) lib/hex/remote_converger.ex:306: Hex.RemoteConverger.verify_input/2
    (hex 2.0.6) lib/hex/remote_converger.ex:52: Hex.RemoteConverger.converge/2
    (mix 1.15.7) lib/mix/dep/converger.ex:133: Mix.Dep.Converger.all/4
    (mix 1.15.7) lib/mix/dep/converger.ex:89: Mix.Dep.Converger.converge/4
    #cell:setup:13: (file)

with an open setup, so I have to assume there is something unique at the network level causing it. I will mark this feature request as closed, since as @ericmj points out, the code should already handle redirects and I was just looking in the wrong place when triaging my error. I will open a new issue if I can get a reliable repro with a more specific synopsis!

hexpm / hex