Azure / azure-functions-dotnet-worker

Azure Functions out-of-process .NET language worker
MIT License
407 stars 167 forks source link

WorkerExtensions.csproj trying to restore from private nuget feed, unsuccessfully #1252

Open 1kevgriff opened 1 year ago

1kevgriff commented 1 year ago

I hope this is the right place for this question.

I have this recurring problem with our local Functions app trying to build. We recently moved to .NET 6 Isolated to support some future product features.

On build, I get this error:

1>C:\Program Files\Microsoft Visual Studio\2022\Professional\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.targets(132,5): warning : No Authorization header detected
1>Retrying 'FindPackagesByIdAsync' for source 'https://nuget.pkg.github.com/{{ ourorg }}/download/microsoft.azure.webjobs.extensions.durabletask/index.json'.
1>Response status code does not indicate success: 401 (Unauthorized).
1>C:\Program Files\Microsoft Visual Studio\2022\Professional\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.targets(132,5): warning : No Authorization header detected
1>C:\Users\kevin\AppData\Local\Temp\l20wov0k.lk5\WorkerExtensions.csproj : error NU1301: Failed to retrieve information about 'Microsoft.Azure.WebJobs.Extensions.DurableTask' from remote source 'https://nuget.pkg.github.com{{ ourorg }}/download/microsoft.azure.webjobs.extensions.durabletask/index.json'.
1>Failed to restore C:\Users\kevin\AppData\Local\Temp\l20wov0k.lk5\WorkerExtensions.csproj (in 5.86 sec).

I'm not sure why WorkerExtensions.csproj is trying to look at our private Nuget feed for that package.

Functions.csproj

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <TargetFramework>net6.0</TargetFramework>
        <AzureFunctionsVersion>v4</AzureFunctionsVersion>
        <OutputType>Exe</OutputType>
    </PropertyGroup>
    <ItemGroup>
        <PackageReference Include="Microsoft.Azure.Functions.Extensions" Version="1.1.0" />
        <PackageReference Include="Microsoft.Azure.Functions.Worker" Version="1.10.0" />
        <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.DurableTask" Version="1.0.0-rc.1" />
        <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Http" Version="3.0.13" />
        <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Storage" Version="5.0.1" />
        <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Timer" Version="4.1.0" />
        <PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="1.7.0" OutputItemType="Analyzer" />
        <PackageReference Include="Microsoft.DurableTask.Generators" Version="1.0.0-preview.1" OutputItemType="Analyzer" />
    </ItemGroup>

    <!-- other stuff removed for brevity -->

</Project>
kshyju commented 1 year ago

Are you having trouble when trying to build/restore packages using Visual studio? Do you have the nuget.org package source (https://api.nuget.org/v3/index.json) enabled?

image

If you are trying to restore using "Manage Nuget packages" tab of the project, do you have "nuget.org" package source selected?

image

ghost commented 1 year ago

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

1kevgriff commented 1 year ago

@kshyju Regular restore works fine. This is only trying to build our project if our private repo is enabled:

image

In the above, I usually have to disable our package source for it to work. nuget.org is typically the default (or all if I'm in the nuget explorer)

jviau commented 1 year ago

@1kevgriff I believe the issue is that part of the build process will create a temporary project and invoke dotnet restore on that. This target is not inheriting Visual Studio auth for the restore operation. Can you try installing Nuget Credential Provider and trying again? You probably need to include the -netfx switch).

madhavireddythirumuru commented 1 year ago

Hi @1kevgriff, I could resolve similar issue by following the below work around steps.

  1. Update package source to use only the public nuget source "https://api.nuget.org/v3/index.json" and not any of the private feeds.
  2. Restore nuget packages and Rebuild the solution/project.
  3. When build is successful, change the package source back to your private feed.
  4. Now build the solution once again. This should solve the problem

One more alternative is :

  1. Create a similar project with same dependencies in a separate solution with package source as public nuget feed.
  2. Build the solution.
  3. Change the package source to private nuget feed and rebuild the solution
  4. Now go the previous solution where this issue happened first and rebuild the solution (with out changing the feed) Build will be successful this time.

I hope these steps would help you resolve the issue.

1kevgriff commented 1 year ago

@1kevgriff I believe the issue is that part of the build process will create a temporary project and invoke dotnet restore on that. This target is not inheriting Visual Studio auth for the restore operation. Can you try installing Nuget Credential Provider and trying again? You probably need to include the -netfx switch).

I installed Nuget Credential Provider but it looks like I'm still having the same issue. I'm not entirely sure if there is another step after the install.

After the install, I removed the private repo and re-added it within VS 2022. But no help.

jonbolwell commented 1 year ago

@1kevgriff I had the same issue and was able to resolve by adding PackageSourceCredentials with a Personal Access Token created from Azure DevOps to the NuGet.config file under C:\Users\username\AppData\Roaming\NuGet

<configuration>
  <packageSources>
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
    <add key="MyOrg" value="https://pkgs.dev.azure.com/xxxxxx/_packaging/MyOrg/nuget/v3/index.json" />
    <add key="Microsoft Visual Studio Offline Packages" value="C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\" />
  </packageSources>
 <packageSourceCredentials>
    <MyOrg>
        <add key="Username" value="... username here ...." />
        <add key="ClearTextPassword" value="... personal access token here ...." />
    </MyOrg>
</packageSourceCredentials>
</configuration>

Adding a PAT I got from here https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=Windows#create-a-pat

Hope that works for you as well

1kevgriff commented 1 year ago

This is my %appdata%/nuget/nuget.config file:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
    <add key="Microsoft Visual Studio Offline Packages" value="C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\" />
    <add key="My Org Name" value="https://nuget.pkg.github.com/MyOrgName/index.json" />
  </packageSources>
  <packageRestore>
    <add key="enabled" value="True" />
    <add key="automatic" value="True" />
  </packageRestore>
  <bindingRedirects>
    <add key="skip" value="False" />
  </bindingRedirects>
  <packageManagement>
    <add key="format" value="0" />
    <add key="disabled" value="False" />
  </packageManagement>
  <packageSourceCredentials>
    <MyOrgName>
        <add key="Username" value="(username)" />
        <add key="Password" value="(token) />
      </MyOrgName>
  </packageSourceCredentials>
</configuration>

It seems like I have everything where it is supposed to be, but VS building WorkerExtensions.csproj on the fly isn't using any of it.

1kevgriff commented 1 year ago

Also noticed this in the build output for the project:

24>C:\Program Files\Microsoft Visual Studio\2022\Professional\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.targets(132,5): warning : The plugin credential provider could not acquire credentials. Authentication may require manual action. Consider re-running the command with --interactive for `dotnet`, /p:NuGetInteractive="true" for MSBuild or removing the -NonInteractive switch for `NuGet`
24>C:\Users\kevin\AppData\Local\Temp\lgyvtjis.242\WorkerExtensions.csproj : error NU1301: Unable to load the service index for source https://nuget.pkg.github.com/MyOrgName/index.json.
jonbolwell commented 1 year ago

It definitely needed a change from Password to ClearTextPassword when I put the token in, it didn't work for me either using the key as just Password

1kevgriff commented 1 year ago

I need to review again, but moving %APPDATA% Nuget.Confg to ClearTextPassword didn't work.

fabiocav commented 1 year ago

Flagging this for investigations and ensure the following scenarios are handled:

fabiocav commented 1 year ago

@1kevgriff as a temporary workaround, moving your private feed entry from the global nuget.config file to a project level nuget.config would prevent the inner build from picking this up and potentially avoid this issue.

jviau commented 1 year ago

Linking this here so it is not lost: this comment also brings up another way in which the nuget restore on the inner build is problematic. Maybe we can address both issues at once.

DuckScapePhilip commented 1 year ago

@1kevgriff as a temporary workaround, moving your private feed entry from the global nuget.config file to a project level nuget.config would prevent the inner build from picking this up and potentially avoid this issue.

This worked for me. I had the Nuget.config in the solution folder, when I copied it into my azure functions project folder with build action set to NONE and Copy to Output Directory set to DO NOT COPY, it worked for me.

jwstauber2 commented 1 year ago

Any updates on this? The only workaround above that worked for was adding the packageSourceCredentials with a PAT to my global NuGet.Config, but this is far from ideal, as now the token is going to expire a year from now and I'm going to hit this issue again and probably forget why, not to mention the fact that we have to educate ALL our developers to do this manually.

dkboon01 commented 12 months ago

I just spent all day struggling with this. Very frustrated because at times I would get it to work by taking out my company's private feed. However, when I have to be in and out of projects . Three hours later go to build again and stopped again. This error message is not helpful a message it doesn't like my private feed for a workerextension.csproj that I can't see in my solution. I got this to finally work a little more consistent by changing the nuget.config to use the generated PAT token . Please fix this error. I use azure functions all the time. This particular project is the first one using the Azure sql input binding. Thank you to everyone that is on this thread sharing about how they solved this problem.

jviau commented 12 months ago

To work around this you can try installing the nuget credential provider and performing an interactive CLI restore:

dotnet restore --interactive
dwalba commented 11 months ago

I ran into this same issue using Visual Studio 2022. I was able to resolve the issue by updating to the latest version of Visual Studio 2022. I then deleted NuGet.Config from %AppData%\Roaming\NuGet\NuGet.Config and then restarted Visual Studio 2022 and everything was building as normal.

eureka-gh commented 11 months ago

Thanks @dwalba, it saves my entire date for this issue. I had exactly the same issue with VS 2022. Also I am using private artifact feed (no public feed, e.g. nuget.org is not allowed). My solution need to have a private Nuget.config file in project root folder. and %AppData%\Roaming\NuGet\NuGet.Config contains my PAT (personal access token) I acquired from Azure DevOps a couple of months ago.

Removing this global Nuget.config seems clean-up my PAT in that file. Then I restarted VS 2022 and rebuilt my solution. No "WorkerExtensions.csproj : error NU1301: Unable to load the service index for source {my private source address}"any more. What confuse me is that the VS 2022 doesn't ask for the credential to access my private artifact feed neither(I would expect re-enter my credential to reset the nuget-restore). Not sure if it works for AzureDevOps pipeline, I'll come back to this thread after testing in AzureDevOps building pipeline.

/////////////////////////////// I can confirm that AzureDevOps pipeline works as well. It looks to me that this error is only in local debugging environment.

Hope this info helps these who are stuck on this issue.

jviau commented 11 months ago

The reason this only occurs in local environment with VS is that in this scenario restore is being performed by two different technologies:

First the outer restore is done by Visual Studio. VS has its own way of resolving credentials, so it gets authenticated. The second inner restore is done by Functions SDK using the MSBuild task. VS is not involved here and thus the credentials do not get resolved.

When performing a CI build, you are most likely authenticating to the nuget feed ahead of time - explicitly or implicitly. So now both outer & inner restores are authenticated.

We will need to work with the VS team to see if there is a way for us to inherit their authentication for our inner restore. Until then, you need to either (1) remove private feeds from restore, or (2) pre-authenticate to that feed via dotnet restore --interactive.

Coder3333 commented 9 months ago

I feel like I have been experiencing this issue for years. The only workaround I have (disabling my private feed, build, then enable the private feed) becomes really tiresome quickly. Is anything being done to allow this to work without the manual intervention?

bandersen22000 commented 8 months ago

I resolved it by downgrading Microsoft.Azure.Functions.Worker.Sdk from 1.16.1 to 1.16.0

DenLilleMand commented 5 months ago

I updated my ADO pipeline to use NuGet Authenticate task before calling restore and then updating the windows image I was running on to Windows Server 2022. This fixed my issue.

steps:
- task: UseDotNet@2
  displayName: 'Use .NET 8.x SDK'
  inputs:
    version: 8.x
- task: NuGetAuthenticate@1
  displayName: 'Nuget Authenticate'
- task: DotNetCoreCLI@2
  displayName: 'dotnet restore'
  inputs:
    command: restore
    projects: ...
    feedsToUse: config
    nugetConfigPath: src/NuGet.Config
- task: DotNetCoreCLI@2
  displayName: 'dotnet build/publish'
  inputs:
    command: publish
    publishWebProjects: false
    projects: ...
    arguments: '/p:Configuration=Release /warnaserror -o $(PackagePublishPath)'
    zipAfterPublish: false
palysalman-cognizant commented 5 months ago

We also had this issue when upgrading from Microsoft.Azure.Functions.Worker.Sdk Version="1.16.4" -> "1.17.0" We currently have NuGet.config which currently has 2 sources in order: private and nuget.org. Everything was fine until we upgraded the package and our pipelines started to fail with the 401 errors. As a workaround we reordered our Nuget.config to first point to nuget.org and only second to our private feed:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="Private" value="https://pkgs.dev.azure.com/..../nuget/v3/index.json" />
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
  </packageSources>
</configuration>
TO
<configuration>
  <packageSources>
    <clear />
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
    <add key="Private" value="https://pkgs.dev.azure.com/..../nuget/v3/index.json" />
  </packageSources>
</configuration>

We came across this commit https://github.com/Azure/azure-functions-dotnet-worker/commit/4089705f849bbd67be41f0e2fdd7777f379eff83, which might have caused the issue. We would have thought that the process to restore would attempt to try all sources in the list, rather than just failing on the first source.

The error we were getting is: Determining projects to restore... Retrying 'FindPackagesByIdAsync' for source 'https://pkgs.dev.azure.com/...private.../_packaging/886ae49c-8c78-4e20-9fca-3f36440e134e/nuget/v3/flat2/microsoft.net.sdk.functions/index.json'. Response status code does not indicate success: 401 (Unauthorized)

eshvatskyi commented 5 months ago

Flagging this for investigations and ensure the following scenarios are handled:

  • Custom extension packages in private feeds only (WebJobs extensions)
  • Authenticated feeds
  • Environment with network restrictions limiting access to NuGet during restore only

For the last one

  • Environment with network restrictions limiting access to NuGet during restore only

I updated my ADO pipeline to use NuGet Authenticate task before calling restore and then updating the windows image I was running on to Windows Server 2022. This fixed my issue.

Afaik the issue is that we didn't have access to NuGet during the build step and because a .csproj file is generated in the build step it wasn't possible to fetch its dependencies.

steps:
- task: UseDotNet@2
  displayName: 'Use .NET 8.x SDK'
  inputs:
    version: 8.x
- task: NuGetAuthenticate@1
  displayName: 'Nuget Authenticate'
- task: DotNetCoreCLI@2
  displayName: 'dotnet restore'
  inputs:
    command: restore
    projects: ...
    feedsToUse: config
    nugetConfigPath: src/NuGet.Config
- task: DotNetCoreCLI@2
  displayName: 'dotnet build/publish'
  inputs:
    command: publish
    publishWebProjects: false
    projects: ...
    arguments: '/p:Configuration=Release /warnaserror -o $(PackagePublishPath)'
    zipAfterPublish: false

works for me in azure pipelines

eugene-rebedailo commented 5 months ago

Same issue here. Decided to downgrade, waiting for the fix.

We also had this issue when upgrading from Microsoft.Azure.Functions.Worker.Sdk Version="1.16.4" -> "1.17.0" We currently have NuGet.config which currently has 2 sources in order: private and nuget.org. Everything was fine until we upgraded the package and our pipelines started to fail with the 401 errors. As a workaround we reordered our Nuget.config to first point to nuget.org and only second to our private feed:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="Private" value="https://pkgs.dev.azure.com/..../nuget/v3/index.json" />
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
  </packageSources>
</configuration>
TO
<configuration>
  <packageSources>
    <clear />
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
    <add key="Private" value="https://pkgs.dev.azure.com/..../nuget/v3/index.json" />
  </packageSources>
</configuration>

We came across this commit 4089705, which might have caused the issue. We would have thought that the process to restore would attempt to try all sources in the list, rather than just failing on the first source.

The error we were getting is: Determining projects to restore... Retrying 'FindPackagesByIdAsync' for source 'https://pkgs.dev.azure.com/...private.../_packaging/886ae49c-8c78-4e20-9fca-3f36440e134e/nuget/v3/flat2/microsoft.net.sdk.functions/index.json'. Response status code does not indicate success: 401 (Unauthorized)

This is not a solution as it defeats the purpose of using the private nuget feed in the first place.

thomasvdb commented 5 months ago

Adding the NuGetAuthenticate task solved the issue for us after upgrading Microsoft.Azure.Functions.Worker.Sdk from 1.14.1 to 1.17.0 using only private NuGet feeds (no link to nuget.org in our setup)

Update: it worked once for some reason but now failing again after triggering a next build

natelaff commented 4 months ago

Also getting this going from Microsoft.Azure.Functions.Worker.Sdk Version="1.16.4" -> "1.17.0". Nuget v3 feed was already above my private feed in NuGet.config. Have not yet found a solution.

iRubens commented 4 months ago

Same here. The only way we actually found to solve it is downgrading from "1.17.0" to "1.16.4". No issues with "1.17.0" on builds without "nuget.config" (with private feeds specified via "vstsFeed" field).

atbez-miview commented 3 months ago

Interestingly I found this thread because upgrading from 1.17.2 to 1.17.3-preview caused this issue in Azure Devops for me. And reverting to 1.17.2 fixed it. I was upgrading to test if it fixed a different issue I was having locally with the generated csproj line:

<Error Condition="'$(TargetFramework)' != 'net6.0'" Text="The target framework '$(TargetFramework)' must be 'net6.0'. Verify if target framework has been overridden by a global property." />

The TargetFramework in my functionapp csproj is set to net8.0, so this errors the build. 1.17.3-preview did resolve this error.

I did notice from the error, that 1.17.3-preview put the generated WorkerExtensions.csproj in a different location than

1.17.3-preview = /home/vsts/work/1/s/{project}/obj/Release/net8.0/WorkerExtensions/WorkerExtensions.csproj

1.17.2 = /tmp/dzkoejrx.wkv/WorkerExtensions.csproj

That seems like it would put the 1.17.2 version outside the influence of the nuget.config in the {project} folder.

jviau commented 3 months ago

Regarding the nuget restore 401 issues when using SDK versions 1.17.0, 1.17.1, and 1.17.3-preview1 (and not 1.17.2), this is because those SDK versions fix an issue where the inner restore was never inheriting the nuget feeds of the outer restore. I believe the source of the 401s may be that you do not have nuget authenticated during build? Depending on how your pipelines are setup, the outer project may be restoring at a different time with the authenticated feed. The fix is to ensure nuget is authenticated (if you use authenticated feeds) during build. As @eshvatskyi mentioned, for Azure Pipelines the fix is to use the NugetAuthenticate@1 task.

As for why we made the change for inner build inheriting nuget feeds: it was considered a bug before-hand. You could have worker extension package A in your outer csproj, which depends on webjobs extension package B as part of the inner restore. We could pull them from different feeds, or worse not find the webjobs extension package at all. This led to inconsistent / broken builds when the package(s) were available in both or only one feed.

jviau commented 3 months ago

@atbez-miview that TFM error is an intentional addition to fail a misconfiguration at build time. Otherwise using the wrong TFM for the inner build can cause a runtime failure. Note that the inner-build TFM is different than your function app csproj's TFM. If you use net8.0, the inner build can still be net6.0 - they are separate builds.

The change in build location was for two reasons:

  1. To inherit nuget.config, as we want to respect usage of non-nuget.org feeds
  2. to enable incremental build (predictable location of WorkerExtensions.csproj, we can re-use it between builds to enable incremental builds)
atbez-miview commented 3 months ago

Regarding the nuget restore 401 issues when using SDK versions 1.17.0, 1.17.1, and 1.17.3-preview1 (and not 1.17.2),

So 1.17.2 had a reversion of the issue? I will go back and check 1.17.0 and 1.17.1. I only ever saw the issue in Devops with 1.17.3-preview1, but I looks like we skipped from 1.16.4 to 1.17.2

I believe the source of the 401s may be that you do not have nuget authenticated during build? Depending on how your pipelines are setup, the outer project may be restoring at a different time with the authenticated feed. The fix is to ensure nuget is authenticated (if you use authenticated feeds) during build. As @eshvatskyi mentioned, for Azure Pipelines the fix is to use the NugetAuthenticate@1 task.

Thats the thing ... We are already using the nuget authentication task, and all of the other dotnet steps in the build (including the dotnet restore) run fine.

As for why we made the change for inner build inheriting nuget feeds: it was considered a bug before-hand. You could have worker extension package A in your outer csproj, which depends on webjobs extension package B as part of the inner restore. We could pull them from different feeds, or worse not find the webjobs extension package at all. This led to inconsistent / broken builds when the package(s) were available in both or only one feed.

Makes sense

@atbez-miview that TFM error is an intentional addition to fail a misconfiguration at build time. Otherwise using the wrong TFM for the inner build can cause a runtime failure. Note that the inner-build TFM is different than your function app csproj's TFM. If you use net8.0, the inner build can still be net6.0 - they are separate builds.

I get that as well. My point in mentioning it is that I seemingly have no control over the input to the error. Since this happens l locally I have tried every version from 1.16.4 up, and 1.17.2 is the only one that throws the error. Literally other than the package version, nothing else in my project changed. (one character, ctrl-s, fail) Seems like something about .2 is setting the $(TargetFramework) value differently than the others, but it was resolved in 1.17.3-preview1

jviau commented 3 months ago

Thats the thing ... We are already using the nuget authentication task, and all of the other dotnet steps in the build (including the dotnet restore) run fine.

Can you share your auth and build steps? With any secrets redacted of course.

I get that as well. My point in mentioning it is that I seemingly have no control over the input to the error. Since this happens l locally I have tried every version from 1.16.4 up, and 1.17.2 is the only one that throws the error. Literally other than the package version, nothing else in my project changed. (one character, ctrl-s, fail) Seems like something about .2 is setting the $(TargetFramework) value differently than the others, but it was resolved in 1.17.3-preview1

The original issue was that TargetFramework when set as a global property would override the TFM in the inner-build. This could happen in something like VS publish. So, we did a two-layer fix: (1) add the error to fail fast and (2) suppress the global property from the inner build.

1.17.2 only had fix 1, and fix 2 was reverted due to other issues with the SDK. Fix 2 should prevent that error from occurring and is present in 1.17.3-preview1. Let me know if you still see that issue in this preview version.

Literally other than the package version, nothing else in my project changed

This means you already had this issue, we just weren't detecting it correctly. You may have ran without issue, but adding a package that happened to have multi-TFM dependencies could have led to runtime assembly load failures.

atbez-miview commented 3 months ago

1.17.2 only had fix 1, and fix 2 was reverted due to other issues with the SDK. Fix 2 should prevent that error from occurring and is present in 1.17.3-preview1. Let me know if you still see that issue in this preview version.

Ok ... so 1.17.2 had a couple reversions. Yes, as I mentioned, 1.17.3-preview1 did fix that particular issue.

The original issue was that TargetFramework when set as a global property would override the TFM in the inner-build. This could happen in something like VS publish.

Question on this. We never had an issue building before 1.17 and we had 7 and then 8 in the global.json. Was the override you are talking about introduced in 1.17?

Can you share your auth and build steps? With any secrets redacted of course.

Sure. We have a dozen or so microservices that all share the same build template ... here are the steps up to failure (build) following that are tests, publish, docker build, push acr, etc ...

        steps:
          - task: UseDotNet@2
            displayName: "Install .NET ${{parameters.dotnetVersion}}"
            inputs:
              packageType: "sdk"
              version: "${{parameters.dotnetVersion}}"

          - task: DotNetCoreCLI@2
            displayName: "Install DotNet-EF"
            inputs:
              command: custom
              custom: tool
              arguments: install --global dotnet-ef --version ${{parameters.efVersion}}

          - script: |
              sudo apt-get update
              sudo apt-get install -y libsqlite3-mod-spatialite libspatialite-dev
            displayName: 'Install test dependencies'

          - checkout: self
            fetchDepth: 1

          - task: Cache@2
            condition: eq(variables.disableNugetCache, false)
            displayName: Restore NuGet cache
            inputs:
              key: 'nuget | "$(Agent.OS)" | **/*.csproj,!**/bin/**'
              restoreKeys: |
                nuget | "$(Agent.OS)"
              path: $(NUGET_PACKAGES)
            continueOnError: true
            env:
              VSO_DEDUP_REDIRECT_TIMEOUT_IN_SEC: 5

          - task: NuGetAuthenticate@1
            displayName: 'NuGet Authenticate'
            inputs:
              nuGetServiceConnections: MyNugetServiceConnection

          - task: DotNetCoreCLI@2
            displayName: Restore NuGet Packages
            inputs:
              command: "restore"
              projects: "$(solution)"
              feedsToUse: 'config'
              nugetConfigPath: '$(serviceDirectory)/nuget.config'

          - task: DotNetCoreCLI@2
            displayName: Build
            inputs:
              command: "build"
              projects: "$(solution)"
              arguments: >
                -c $(buildConfiguration) --no-restore
                -p:SatelliteResourceLanguages=en

We have reverted to 1.16.4 for now with no issues. We had no specific reason to go to 1.17, just a developer that was keeping packages current in a PR.

jviau commented 3 months ago

Question on this. We never had an issue building before 1.17 and we had 7 and then 8 in the global.json. Was the override you are talking about introduced in 1.17?

The overriding was present in all Worker SDK versions except for 1.17.* (but also present in 1.17.2). This does not have to do with global.json - that pins the SDK used. The issue is related to TargetFramework. 1.17.* added a detection (via a build error) for this problem. So only with package version 1.17.2 could override occur and be detected, leading to a build failure (which is better than a potentially obscure runtime failure).

Are you experiencing restore issues with that build, or only the TFM issues?

atbez-miview commented 2 months ago

So only with package version 1.17.2 could override occur and be detected

Ok ... so pre 1.17 was overriding, but not causing in problems for ME ... but sometimes the override was causing issues for others? fair.

Are you experiencing restore issues with that build, or only the TFM issues?

Yes. 1.17.* (excluding .2) are building the generated project under the /bin folder therefore using our nuget.config pointing to our private repo. I understand this is desired behavior. The problem for us being that the generated project is throwing 401 during the restore. All of the other dotnet commands work fine, including the dotnet restore previous to the build. I'm starting to wonder if the --no-restore on the dotnet build step is somehow preventing the generated project from pulling the needed auth. I will have to test that at some point.

jviau commented 2 months ago

@atbez-miview could you collect an msbuild binlog? https://msbuildlog.com/

That may give us insight into what is happening in that failing restore task

danorthogonal commented 2 months ago

I ran into this same issue using Visual Studio 2022. I was able to resolve the issue by updating to the latest version of Visual Studio 2022. I then deleted NuGet.Config from %AppData%\Roaming\NuGet\NuGet.Config and then restarted Visual Studio 2022 and everything was building as normal.

This worked for me too. VS2022 when updating Azure function from .NET6. Extremely frustrating. Thanks @dwalba

cpsaez commented 2 months ago

Some problem here and I've fixed this updating Azure.Functions.Worker.Sdk from 1.17.0 to 1.17.2 and Azure.Functions.Worker from 1.21.0 to 1.22.0. Hope this help

DaveMcCrady commented 1 month ago

Some problem here and I've fixed this updating Azure.Functions.Worker.Sdk from 1.17.0 to 1.17.2 and Azure.Functions.Worker from 1.21.0 to 1.22.0. Hope this help

Thank you! This resolved the same issue building in Azure Pipelines for me. D:\a\1\s\obj\Release\net8.0\WorkerExtensions\WorkerExtensions.csproj : error NU1301: Unable to load the service index for source { ...private nuget feed... }

briandunnington commented 4 days ago

So only with package version 1.17.2 could override occur and be detected

Ok ... so pre 1.17 was overriding, but not causing in problems for ME ... but sometimes the override was causing issues for others? fair.

Are you experiencing restore issues with that build, or only the TFM issues?

Yes. 1.17.* (excluding .2) are building the generated project under the /bin folder therefore using our nuget.config pointing to our private repo. I understand this is desired behavior. The problem for us being that the generated project is throwing 401 during the restore. All of the other dotnet commands work fine, including the dotnet restore previous to the build. I'm starting to wonder if the --no-restore on the dotnet build step is somehow preventing the generated project from pulling the needed auth. I will have to test that at some point.

@atbez-miview did you ever verify if the --no-restore was the problem? we have the same set up (shared DevOps build scripts) with similar steps and getting the same 401 behavior on the generated project when updating from 1.17.2 -> 1.17.4

Porges commented 3 days ago

There's another problem here which is that WorkerExtensions.csproj is subverting lockfiles and --no-restore. Even if I build with dotnet publish --no-restore it is pulling packages which are not mentioned anywhere in any lockfile. This makes it very hard to review what actual versions of packages are being included in the final build.

Edit: I found that this is already tracked over at #1864 and #1888

bondarenkod commented 2 hours ago

Could you test my workaround? You can find it at this link https://gist.github.com/bondarenkod/4b6a9bc7224e6c8ab09ae113b68de1f5

@Porges