Private repository releases support

[akramhussein]

This issue was covered https://github.com/philss/rustler_precompiled/issues/26 but I was not able to get it to work so wanted to double check if the solution is still correct and if you have any advice:

When hosting a Rust dependency in a private repository and using rustler_precompiled, if there is no plans to publish to hex package how can the downstream Mix project fetch the release as the base_url is private and not authenticaiton details are passed through?

Our rustler_precompiled setup has a correct release + checksum-* file etc and works when the project is made public but not if private. The same downstream project is able to pull from the private repo using a GitHub token and compile it but just not rustler_precompiled.

My understanding from a quick glance at the GitHub docs is that rustler_precompiled would need to pass the GitHub Personal Access Token (PAT) through and use the API. This StackOverflow answer also covers it.

[sliiser]

I am interested in picking this up. @philss, do you have any thoughts on the best approach to take? Any specific config fields you have in mind or should I just come up with something myself?

[philss]

I think we could accept a tuple of {base_url, request_headers}. Maybe we could also accept {Module, :function} to dynamically call a function that can return this tuple. WDYT?

[sliiser]

{base_url, request_headers} would make it a lot more flexible, but not sufficient for private repos due to the asset download URLs in the API not being in the format #{base_url}/#{file_name}. There is an API endpoint that will return a list of assets that will have to be iterated through to find the correct file name from the body and then take the actual URL from that response which will be in the format of https://api.github.com/repos/NAME/REPO/releases/assets/1275759, which will then redirect to S3.

I propose accepting a function (for example in the format {Module, :function}) that would take the name of the binary (with or without the tar.gz suffix) as an argument and return {full_url, request_headers}, not {base_url, headers}. Another option would be to have the function make the request and return the binary content.

However, an issue with accepting {Module, :function} is that during mix rustler_precompiled.download YourRustlerModule, the application code has not been loaded (and the NIF module cannot be loaded), so this function might be unavailable. Thoughts?

I also found an issue with following redirects to S3. :httpc unlike other HTTP clients like curl passes the Authorization header to the new URL after a redirect. This then fails in S3 due to S3 complaining that multiple authorization methods were used (example). However, if {full_url, request_headers} is returned, that can already be the S3 URL and this issue can be bypassed.

[philss]

@sliiser I like your proposal. I think believe my PR fixes the problem of the app being loaded: https://github.com/philss/rustler_precompiled/commit/a5cd67e81ed9dc514cc0ac58f9643eb4e2637f15 WDYT?

[sliiser]

Alright, I'll get on it.