No documentation for 'bring your own access token' with NuGet client libraries

[idg10]

I am trying to use the NuGet client SDK to access a protected feed (hosted on Azure DevOps). I need to perform some operations that aren't directly supported by existing NuGet apps or ADO build pipeline tasks, which is why I'm programming directly against the SDK. The relevant code will be running as part of a build pipeline. It's not entirely clear how NuGet clients are supposed to authenticate in this scenario, but having asked some ALM MVPs, they've pointed me at the system.accesstoken build variable.

If I program directly against the NuGet REST API with HTTP requests, this works fine—I'm able to get the token from that build variable, and it works just fine for accessing the protected feed. I just need to put the token into the HTTP request's Authorization header, prefixed with Bearer, and the ADO package feed is happy to talk to me.

But I don't want to work directly at the HTTP level. I was hoping to use the NuGet client SDK, because it has a whole load of well thought out and extensively tested support for working with NuGet feeds.

Unfortunately, the NuGet Client SDK docs appear to consist entirely of "Go read these three blog entries, and then because those were written in 2016 and there have been important changes since then, meaning that what you've just read is out of date, now read this fourth blog entry."

Here's my starting point:

SourceRepository source = Repository.Factory.GetCoreV3("https://api.nuget.org/v3/index.json");
ListResource listResource = source.GetResource<ListResource>();
IEnumerableAsync<IPackageSearchMetadata> x = await listResource.ListAsync(
    "Endjin.Retry",
    prerelease: false,
    allVersions: true,
    includeDelisted: false,
    log: new NullLogger(),
    token: CancellationToken.None);

IEnumeratorAsync<IPackageSearchMetadata> e = x.GetEnumeratorAsync();
while (await e.MoveNextAsync())
{
    IPackageSearchMetadata c = e.Current;
    Console.WriteLine(c.Identity);
}

This works well enough against the public NuGet.org feed (although given the state of the documentation I've no idea whether this is the best way to do this). It searches for the publicly listed Endjin.Retry package. It finds it. This is good.

However, if I replace the URL there with the one for my private feed: "https://pkgs.dev.azure.com/IanGriffiths/_packaging/Ian.LibraryVersioningExperiment/nuget/v3/index.json" I get this error:

System.AggregateException: One or more errors occurred. (Unable to load the service index for source https://pkgs.dev.azure.com/IanGriffiths/_packaging/Ian.LibraryVersioningExperiment/nuget/v3/index.json.) ---> NuGet.Protocol.Core.Types.FatalProtocolException: Unable to load the service index for source https://pkgs.dev.azure.com/IanGriffiths/_packaging/Ian.LibraryVersioningExperiment/nuget/v3/index.json. ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 401 (Unauthorized).

This seems to occur inside the call to source.GetResource<ListResource>().

This is unsurprising of course—this is a protected feed, and my code has not yet made any attempt to supply credentials, so you'd expect it to fail.

But the question is: how am I supposed to supply a token?

It looks like HttpHandlerResourceV3Provider.cs is where it sets up the HttpClientHandler and HttpMessageHandler. In particular, I found at https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.Protocol/HttpSource/HttpHandlerResourceV3Provider.cs#L62 this code:

messageHandler = new StsAuthenticationHandler(packageSource, TokenStore.Instance)
{
    InnerHandler = messageHandler
};

This made me wonder if I needed to supply the token store with my token. So I tried adding this line before doing anything else with the client libraries:

TokenStore.Instance.AddToken(new Uri(FeedUrl), token);

But it made no difference. And although I can't claim to have understood how 'STS Authentication' fits into this code, it appears it sets the X-NuGet-STS-Token header. But the token I've got is for use as an HTTP Authorization Bearer header. In case it's relevant, here's the decoded body of the JWT that I've got:

{
  "nameid": "fb6c9195-1108-490b-a5a2-1d36f69972af",
  "scp": "app_token",
  "aui": "1e7bc101-05e3-4396-8ae2-481fd28eed86",
  "sid": "f3614a90-e4df-4cf5-ad12-f2f3f0c14fa8",
  "iss": "app.vstoken.visualstudio.com",
  "aud": "app.vstoken.visualstudio.com|vso:49ebc6d2-c78c-4aa6-96af-1a4273e6143b",
  "nbf": 1548066236,
  "exp": 1548071036
}

To reiterate, this works just fine against the package feed when I hit it manually from Postman. (Obviously I need to get a new token every now and then because they expire, but I've manually verified that the token I'm trying to use with the NuGet Client SDK is still valid even after seeing 401 failures in my app by using it again directly against the HTTP API.)

I've cloned the NuGet/NuGet.Client repo and searched it for use of the Authorization header, and the only place it shows up directly is in unit tests, all of which appear to be looking for basic auth. So it appears that the client doesn't set the Authorization header directly. As far as I can tell, instead it sets the HttpClientHandler.Credentials property with an HttpSourceCredentials. This seems to end up providing a NetworkCredential, and it's not clear whether it's even possible to set the Authorization header to a Bearer token with one of those. Pretty much every example out there showing how to use bearer tokens in .NET does it by setting the Headers.Authorization property on the HttpRequestMessage.

So the 'obvious' mechanisms for that would be either to use the HttpClient.DefaultRequestHeaders, or to insert an HttpMessageHandler, but it looks like the creation of the client and handler chain is all locked up inside HttpHandlerResourceV3Provider and is not open to extension.

I took a look at https://github.com/Microsoft/artifacts-credprovider too, and from that I get the impression that it doesn't actually set a Bearer token either.

At this point I dredged up a dim memory that with ADO APIs, you can also use basic auth, with anything you like as the username and a PAT as the password, so I tried this:

private class Creds : ICredentialProvider
{
    private readonly string token;

    public Creds(string token)
    {
        this.token = token;
    }

    public string Id { get; }

    public Task<CredentialResponse> GetAsync(Uri uri, IWebProxy proxy, CredentialRequestType type, string message, bool isRetry, bool nonInteractive, CancellationToken cancellationToken)
    {
        return Task.FromResult(new CredentialResponse(new NetworkCredential("x", this.token)));
    }
}

and also this:

var ps = new AsyncLazy<IEnumerable<ICredentialProvider>>(() => Task.FromResult<IEnumerable<ICredentialProvider>>(new ICredentialProvider[] { new Creds(token) }));
HttpHandlerResourceV3.CredentialService = new Lazy<ICredentialService>(() => new CredentialService(ps, true, true));

This has in fact enabled me to move forwards. But I'm left with a feeling of "Surely this isn't what we're actually expected to do?"

Details about Problem

NuGet product used: NuGet Client SDK (NuGet.Credentials, NuGet.Protocol)

Package versions: 4.9.2

OS version: Windows 10 v1809 (17763.253)

Tagging @rrelyea since he offered to help with this on Twitter

[cpyfferoen]

@idg10 with your excellent issue post i was able to move forward on my own project. I posted my API (unpolished, but functional) as a gist https://gist.github.com/cpyfferoen/74092a74b165e85aed5ca1d51973b9d2

THANK YOU for providing the hints I needed to get me over this hill of ridiculously sparse documentation/examples

[nkolev92]

Issue moved to NuGet/docs.microsoft.com-nuget #1987 via ZenHub