Allow using hex.pm mirrors!

[massivefermion]

Using hex.pm mirrors would be useful in situations where hex.pm itself is not accessible, like for users in countries under sanctions!

[lpil]

Let's make this a priority.

[lpil]

Shayan says this would not be sufficient to help here as the Hex API is still not accessible.

[massivefermion]

There are basically two APIs that are used here, first for getting the package metadata(hex.pm/api) and second for downloading the package itself(repo.hex.pm). The first one is the problem and ironically that's the one that does not have a mirror listed in the link above (we'll cross that bridge when we get to it I guess!).

Both of those are hard coded here. At some place(either in the gleam compiler or hexpm-rust) it should be checked that whether another base URI is provided by the user(e.g. using an environmental variable) and if it is, the provided base URI should be used. It seems to me it makes more sense to do it in hexpm-rust in the new function linked above.

There is also the question that whether we should allow the user to provide base URIs for both of them or only hex.pm/api. It seems to me now that we're doing this, it makes more sense to enable it for both of them just in case.

I'm going to work on this as soon as I have definite answers to the above questions.

P.S. Just to be comprehensive, I should add that repo.hex.pm could be used in place of hex.pm/api for getting the package metadata but the workflow would be so much different that it does not make sense to change it at this point or for this reason.

[Pi-Cla]

What is the status on this issue? I would be interested in trying to cook a PR if everyone else is busy

[lpil]

We determined that this wasn't enough for massivefermion to use Hex (as it doesn't solve API access) so we did not continue. We can make this more urgent if it is useful to someone.

[artman41]

Hi,

Would it be possible to introduce a config within the toml to specify the Hex mirror to pull deps from at least?

With the environment I'm working in, we can't access deps from an external source - instead, we have a security team download files for us which we upload to an internal Hex server.

Ideally, what I'm after is functionality within Gleam itself to specify the mirror to pull deps from rather than needing to write a makefile to manually clone deps and then add the deps as Local paths within the toml - it's a lot of effort for something that should be relatively streamline :)

[lpil]

Given we use the Hex API to pull metadata that be enough to work in your situation?

[artman41]

Given we use the Hex API to pull metadata that be enough to work in your situation?

I think so?

I've more experience with Erlang than Gleam so far so you'll have to forgive me for not being 100% clued up on the hex stuff 😉 but in my experience using erlang.mk, it's generally been as easy as swapping the https://repo.hex.pm url for our own Hex mirror

Since we use a Hex instance internally, I'm sure the api would work with what you guys currently do 👍

[lpil]

Are you sure? Why is it that you can make requests to the API but not to the repo on your work network?

[artman41]

Are you sure? Why is it that you can make requests to the API but not to the repo on your work network?

ah probably me not explaining myself fully - we redirect all calls externally to the internal mirror

so api calls and such will all go to our internal instance 👍

Just need a way of allowing gleam to contact a mirror rather than the official Hex instance 😄

[lpil]

So you run a mirror of the package repo and then some sort of proxy for the API? Or do you run an entirely new instance of the Hex web application?

[artman41]

So you run a mirror of the package repo and then some sort of proxy for the API? Or do you run an entirely new instance of the Hex web application?

entirely new instance of the Hex app 👍

[lpil]

Fantastic, thank you.

HEX_CDN_URL and HEX_API_URL for the variables perhaps?

[artman41]

Fantastic, thank you.

HEX_CDN_URL and HEX_API_URL for the variables perhaps?

that would be brilliant ❤️

If possible, I'd ideally like an option in the gleam.toml to set it across the project but I'll take what I can get 😆

[lpil]

gleam.toml is for project configuration and does not today contain anything specific to the compilation environment being used at the time, so I don't think it's the right place for it. I could be persuaded otherwise by some prior art in other build tools perhaps.

[chvish]

@lpil Rebar (the Erlang build tool) allows overriding repo urls (typically used for self hosted/mirrored hex.pm instances). You can specify that both on a project level rebar.config, as well as in a global config (stored in ~/.config for example).

[TinyLittleWheatley]

Any decisions on config file or environment variables? I personally don't care either way but really could use the feature🙃

[lpil]

We decided on environment variables but no one has picked up the work I'm afraid.