search

microsoft / artifacts-credprovider

The Azure Artifacts Credential Provider enables dotnet, NuGet.exe, and MSBuild to interactively acquire credentials for Azure Artifacts feeds.
MIT License
759 stars 707 forks source link

error NU1301: Unable to load the service index for source #446

Closed solidcloudio closed 1 year ago

solidcloudio commented 1 year ago

Regardless of what I try here I get: error NU1301: Unable to load the service index for source. I'm able to get this to run on a different server that uses linux containers, but on a Windows build machine I get a failure to connect to the feed.

I'm using this in the context of Azure Devops and a Multi-Stage Docker build in Windows I've defined a pipeline variable to store my PAT token. The relevant build section:

     - task: NuGetAuthenticate@1
      displayName: 'NuGet Authenticate'
    - task: Docker@2
      displayName: Build docker image
      inputs:
        command: build
        containerRegistry: $(dockerRegistryServiceConnection)
        dockerfile: $(dockerfilePath)
        repository: $(imageRepository)
        tags: |
          $(tag)
        arguments: --build-arg FEED_ACCESSTOKEN=$(PAT)

My nuget.config

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="NugetServer" value="https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json" />
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
  </packageSources>
</configuration>

Then in my dockerfile:

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build

SHELL ["pwsh", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
USER ContainerAdministrator
RUN Invoke-WebRequest https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.ps1 -OutFile installcredprovider.ps1;
RUN .\installcredprovider.ps1 -AddNetFx;
RUN del installcredprovider.ps1
USER ContainerUser

WORKDIR /src

COPY ["nuget.config", "./"]
COPY ["Service.Test.csproj", "Service.Test/"]
ARG FEED_ACCESSTOKEN
ENV VSS_NUGET_EXTERNAL_FEED_ENDPOINTS="{`"endpointCredentials`": [{`"endpoint`":`"https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json`", `"username`":`"docker`", `"password`":`"${FEED_ACCESSTOKEN}`"}]}"
ENV NUGET_PLUGIN_ENABLE_LOG='true'
ENV NUGET_PLUGIN_LOG_DIRECTORY_PATH='C:\logs'
RUN echo $Env:VSS_NUGET_EXTERNAL_FEED_ENDPOINTS

RUN dotnet restore "Service.Test.csproj" --configfile "nuget.config" --verbosity quiet

This just fails with:


Step 1/37 : FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
 ---> b88f7cdde9c4
Step 2/37 : WORKDIR /app
 ---> Running in 867dbf4d2b4e
Removing intermediate container 867dbf4d2b4e
 ---> 5018948e3063
Step 3/37 : FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
Step 4/37 : SHELL ["pwsh", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
 ---> Running in 7206fe313eaa
Removing intermediate container 7206fe313eaa
 ---> bf8e39e2a4dd
Step 5/37 : USER ContainerAdministrator
 ---> Running in ae34724362b5
Removing intermediate container ae34724362b5
 ---> f2a1b93b9169
Step 6/37 : RUN Invoke-WebRequest https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.ps1 -OutFile installcredprovider.ps1;
 ---> Running in 649a8147f221
Removing intermediate container 649a8147f221
 ---> 30af9cc22fc8
Step 7/37 : RUN .\installcredprovider.ps1 -AddNetFx;
 ---> Running in 3f7439748466
Fetching release https://api.github.com/repos/Microsoft/artifacts-credprovider/releases/latest

    Directory: C:\Windows\TEMP

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d----           8/15/2023  6:09 AM                CredProviderZip
Downloading https://github.com/microsoft/artifacts-credprovider/releases/download/v1.0.9/Microsoft.NuGet.CredentialProvider.zip to C:\Windows\TEMP\CredProviderZip\Microsoft.NuGet.CredentialProvider.zip
Extracting zip to the Credential Provider temp directory C:\Windows\TEMP\CredProviderZip
Fetching release https://api.github.com/repos/Microsoft/artifacts-credprovider/releases/latest
d----           8/15/2023  6:09 AM                CredProviderZip
Downloading https://github.com/microsoft/artifacts-credprovider/releases/download/v1.0.9/Microsoft.Net6.NuGet.CredentialProvider.zip to C:\Windows\TEMP\CredProviderZip\Microsoft.Net6.NuGet.CredentialProvider.zip
Extracting zip to the Credential Provider temp directory C:\Windows\TEMP\CredProviderZip
Credential Provider installed successfully

Removing intermediate container 3f7439748466
 ---> 6af881db4df7
Step 8/37 : RUN del installcredprovider.ps1
 ---> Running in 9150ece568b0
Removing intermediate container 9150ece568b0
 ---> 389ce1b3411f
Step 9/37 : USER ContainerUser
 ---> Running in b90819fb584b
Removing intermediate container b90819fb584b
 ---> 253ea5a0856c
Step 10/37 : WORKDIR /src
 ---> Running in 4f51e18d470f
Removing intermediate container 4f51e18d470f
 ---> bc67561f1d1b
Step 11/37 : COPY ["nuget.config", "./"]
 ---> 5d7adeb3e202
Step 12/37 : COPY ["Service.Test.csproj", "/"]
 ---> 0c49cf650e6d

 ---> 0c49cf650e6d
Step 13/37 : ARG FEED_ACCESSTOKEN
 ---> Running in 2443745b2b18
Removing intermediate container 2443745b2b18
 ---> 66eb929ae03a
Step 14/37 : ENV VSS_NUGET_EXTERNAL_FEED_ENDPOINTS="{`"endpointCredentials`": [{`"endpoint`":`"https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json`", `"username`":`"docker`", `"password`":`"${FEED_ACCESSTOKEN}`"}]}"
 ---> Running in c9a68a514cab
Removing intermediate container c9a68a514cab
 ---> 0dc18c03bbf7
Step 15/37 : ENV NUGET_PLUGIN_ENABLE_LOG='true'
 ---> Running in 7421994f881e
Removing intermediate container 7421994f881e
 ---> 8ca021909b75
Step 16/37 : ENV NUGET_PLUGIN_LOG_DIRECTORY_PATH='C:\logs'
 ---> Running in c6784b4aa90a
Removing intermediate container c6784b4aa90a
 ---> a925e5d3f37c
Step 17/37 : RUN echo $Env:VSS_NUGET_EXTERNAL_FEED_ENDPOINTS
 ---> Running in 3ebdf9913c5f
{`endpointCredentials`: [{`endpoint`:`https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json`, `username`:`docker`, `password`:`***`}]}
Removing intermediate container 3ebdf9913c5f
 ---> e17e4a5c9a3a
Step 18/37 : RUN dotnet restore "Service.Test/Service.Test.csproj" --interactive --configfile "Service.Test/nuget.config" --verbosity quiet
 ---> Running in db3ea823caa7
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
C:\src\Service.Test\Service.Test.csproj : error NU1301: Unable to load the service index for source https://pkgs.dev.azure.com/{removed}/{project}/_packaging/{removed}/nuget/v3/index.json.
The command 'pwsh -Command $ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue'; dotnet restore "Service.Test/Service.Test.csproj" --interactive --configfile "Service.Test/nuget.config" --verbosity quiet' returned a non-zero code: 1
##[error]The process 'C:\Windows\system32\docker.exe' failed with exit code 1
Finishing: Build docker image

Is there some way to troubleshoot the login? The error message is pretty generic.: error NU1301: Unable to load the service index for source

JohnSchmeichel commented 1 year ago

Was able to repro this, investigating now... might be related to https://github.com/dotnet/dotnet-docker/issues/4803

JohnSchmeichel commented 1 year ago

Ok, the issue is that when installing the credential provider, it first elevates via USER ContainerAdministrator which only installs the credential provider as ContainerAdministrator so the build running as ContainerUser doesn't have access to use the provider to give credentials.

Will come up with a proper fix here... assuming this used to work, but aside from changes to the folder structure and permissions in the base image don't see how this could ever work.

JohnSchmeichel commented 1 year ago

Have few workarounds here until this can be fixed and the sample updated (note this issue only exists on Windows Nano Server containers as it runs as ContainerUser by default):

Option 1

Move the USER ContainerUser instruction after the RUN dotnet ... instructions. This runs the build as ContainerAdministrator which will be able to see and use the installed credential provider. This may or may not be desirable.

Option 2

Use the following docker instructions to grant the additional permissions required to run the credential provider script as ContainerUser. This also may not be desirable.

# Grant BUILTIN\Users access to read & delete from the TEMP directory
USER ContainerAdministrator
RUN & icacls C:\Windows\Temp\ /grant *S-1-5-32-545:'(OI)(CI)(RD,DE,DC)'
USER ContainerUser

# Install the cred provider
WORKDIR /Windows/Temp
RUN Invoke-WebRequest https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.ps1 -OutFile installcredprovider.ps1; `
    .\installcredprovider.ps1; `
    del installcredprovider.ps1
solidcloudio commented 1 year ago

Thanks for the follow-up, I was thinking about that. I read somewhere else about a similar issue when switching user contexts. I ended up just ditching the multi-stage build. The main build script is building it anyways, why build it twice? I can build publish with the main build and just use the docker image as the runtime wrapper. If I copy the publish output from the primary machine and then launch docker with the context refencing the publish output folder, I don't have to build it in docker at all.