`AlternativeRegistry` and authentication

[jonathanstrong]

hello,

As part of working on docs.rs-like functionality for Shipyard.rs, I have been diving into rustwide -- and it's an awesome library! Many thanks to the authors for releasing a great tool.

I was hoping to discuss a couple issues relating to authentication for private/alternative registries, one an early heads up, the other a question.

1) the PR to implement RFC 3139 is currently under review, and it will allow registries to set auth-required: true in the config.json files in their registry indexes, prompting cargo to send an Authorization header (and token) along with crate download requests.

The code in rustwide currently does a good job handling private/alternative registries (IMO), but does not contemplate needing to provide an auth token to fetch .crate files from the registry server.

E.g. download code in src/crates/registry.rs looks like

        workspace
            .http_client()
            .get(&self.fetch_url(workspace)?) 
            .send()?
            .error_for_status()?
            .write_to(&mut BufWriter::new(File::create(&local)?))?;

Also...

pub struct AlternativeRegistry {
    registry_index: String,
    key: Option<String>,
}

... AlternativeRegistry has no way to store an auth token in relation to a private registry.

I'd be happy to work on a PR to facilitate authenticated downloads, the changes seem like they would be pretty straightforward, although the issue might not be quite ripe yet.

2) A general question regarding how to securely "send" sensitive data to the container/sandbox environment that is executing cargo rustdoc:

I see how AlternativeRegistry is built to allow providing an ssh key to authenticate cloning the crate index repository, and I can imagine something similar for auth tokens, but I am less clear on how the ssh-key/token should be passed into the sandbox environment (and removed afterwards).

I haven't been able to find any examples of AlternativeRegistry::authenticate_with_ssh_key being used in a production system, so if you know of any, by all means point me in that direction. And this may actually be a very simple question, but is there any mechanism for temporarily providing a workspace some credentials (like a ssh-key)? In a multi-tenant context, should separate workspace dirs be used for each tenant? (It does seem like the execution time of the initial run, with rustup install, etc. is significant enough to want to avoid repeating it for every crate version, if possible).

Relatedly, in terms of big picture architecture, I am trying to understand where the workspace/sandbox runs. Is it itself inside a container (and then spawns an inner container (Sandbox) to execute cargo rustdoc)? Are the docs.rs instructions for setting up a LXC container the best way to do this? The only way? Outdated and replaced by a new practice? Any guidance on how the components fit together in a big picture sense would be greatly appreciated.

[pietroalbini]

Hello! I don't have too much time to maintain rustwide lately, but I can provide some pointers :slightly_smiling_face:

I'd be happy to work on a PR to facilitate authenticated downloads, the changes seem like they would be pretty straightforward, although the issue might not be quite ripe yet.

More than happy to accept PRs adding new methods to AlternateRegistry to configure it, like authenticate_with_token.

A general question regarding how to securely "send" sensitive data to the container/sandbox environment that is executing cargo rustdoc.

Preparation steps like fetching dependencies from the registry is done outside the sandbox though, in prepare.rs. If you provide the credentials just to cargo fetch (maybe through an environment variable?) the code inside the sandbox won't have access to them.

it itself inside a container (and then spawns an inner container (Sandbox) to execute cargo rustdoc)? Are the docs.rs instructions for setting up a LXC container the best way to do this? The only way? Outdated and replaced by a new practice? Any guidance on how the components fit together in a big picture sense would be greatly appreciated.

The rustwide sandbox is built using ephemeral Docker containers, that docs.rs guide is outdated. If Docker is installed things should just work(tm), rustwide takes care of downloading the image and spawning the containers.

[jonathanstrong]

More than happy to accept PRs adding new methods to AlternateRegistry to configure it, like authenticate_with_token.

cool - I will work something up in the next couple of weeks.

Preparation steps like fetching dependencies from the registry is done outside the sandbox though

ok, looks like I need to get a better understanding of what happens inside the sandbox vs. outside of it. thanks for pointing me in the right direction!