dotnet / sdk

Core functionality needed to create .NET Core projects, that is shared between Visual Studio and CLI
https://dot.net/core
MIT License
2.71k stars 1.07k forks source link

Unable to find fallback package folder #9180

Closed CIPop closed 4 years ago

CIPop commented 6 years ago

Steps to reproduce

  1. Start with a clean Windows box and install Visual Studio Enterprise 2017 (latest) with .NET Core.
  2. Clone https://github.com/azure/azure-iot-sdk-csharp and run jenkins\windows_csharp.cmd in root.

Expected behavior

Builds works as expected.

Actual behavior

BUILD: --- SecurityProvider for TPM Debug ---
Microsoft (R) Build Engine version 15.6.82.30579 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

C:\Program Files\dotnet\sdk\2.1.101\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(172,5): error MSB4018: The "ResolvePackageDependencies" task failed unexpectedly. [S:\cs\security\tpm\src\Microsoft.Azure.Devices.Provisioning.Security.Tpm.csproj]
C:\Program Files\dotnet\sdk\2.1.101\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(172,5): error MSB4018: NuGet.Packaging.Core.PackagingException: Unable to find fallback package folder 'C:\Users\cipop\.dotnet\NuGetFallbackFolder'. [S:\cs\security\tpm\src\Microsoft.Azure.Devices.Provisioning.Security.Tpm.csproj]
C:\Program Files\dotnet\sdk\2.1.101\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(172,5): error MSB4018:    at NuGet.Packaging.FallbackPackagePathResolver..ctor(String userPackageFolder, IEnumerable`1 fallbackPackageFolders) [S:\cs\security\tpm\src\Microsoft.Azure.Devices.Provisioning.Security.Tpm.csproj]
C:\Program Files\dotnet\sdk\2.1.101\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(172,5): error MSB4018:    at Microsoft.NET.Build.Tasks.NuGetPackageResolver.CreateResolver(LockFile lockFile, String projectPath) [S:\cs\security\tpm\src\Microsoft.Azure.Devices.Provisioning.Security.Tpm.csproj]
C:\Program Files\dotnet\sdk\2.1.101\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(172,5): error MSB4018:    at Microsoft.NET.Build.Tasks.ResolvePackageDependencies.get_PackageResolver() [S:\cs\security\tpm\src\Microsoft.Azure.Devices.Provisioning.Security.Tpm.csproj]
C:\Program Files\dotnet\sdk\2.1.101\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(172,5): error MSB4018:    at Microsoft.NET.Build.Tasks.ResolvePackageDependencies.ResolvePackagePath(LockFileLibrary package) [S:\cs\security\tpm\src\Microsoft.Azure.Devices.Provisioning.Security.Tpm.csproj]
C:\Program Files\dotnet\sdk\2.1.101\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(172,5): error MSB4018:    at Microsoft.NET.Build.Tasks.ResolvePackageDependencies.GetPackageAndFileDefinitions() [S:\cs\security\tpm\src\Microsoft.Azure.Devices.Provisioning.Security.Tpm.csproj]
C:\Program Files\dotnet\sdk\2.1.101\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(172,5): error MSB4018:    at Microsoft.NET.Build.Tasks.TaskBase.Execute() [S:\cs\security\tpm\src\Microsoft.Azure.Devices.Provisioning.Security.Tpm.csproj]
C:\Program Files\dotnet\sdk\2.1.101\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(172,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() in E:\A\_work\82\s\src\Build\BackEnd\TaskExecutionHost\TaskExecutionHost.cs:line 631 [S:\cs\security\tpm\src\Microsoft.Azure.Devices.Provisioning.Security.Tpm.csproj]
C:\Program Files\dotnet\sdk\2.1.101\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(172,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__25.MoveNext() in E:\A\_work\82\s\src\Build\BackEnd\Components\RequestBuilder\TaskBuilder.cs:line 787 [S:\cs\security\tpm\src\Microsoft.Azure.Devices.Provisioning.Security.Tpm.csproj]

Environment data

dotnet --info output:

dotnet --info
.NET Command Line Tools (2.1.101)

Product Information:
 Version:            2.1.101
 Commit SHA-1 hash:  6c22303bf0

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.16299
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.1.101\

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.6
  Build    : 74b1c703813c8910df5b96f304b0f2b78cdf194d

There is something interesting in the Users folder:

USERNAME=cipop
USERPROFILE=C:\Users\cipop.CORP

> dir c:\Users\ /b
Administrator
cipop
cipop.CORP
Public

Seems to be related to dotnet/sdk#8578, dotnet/sdk#8536, dotnet/sdk#8578, dotnet/sdk#8680 .

CIPop commented 6 years ago

Workaround:

mkdir C:\Users\cipop\.dotnet\NuGetFallbackFolder

livarcocc commented 6 years ago

I just tried building the repo you mentioned above and the build succeeded, though with test failures.

Can you check your Nuget.Config files and check that you don't have an entry pointing to that folder anywhere?

We had the CLI using that folder a long time ago, before we even released a 2.0 CLI but we have long since moved away from using or setting that folder.

CIPop commented 6 years ago

Can you check your Nuget.Config files and check that you don't have an entry pointing to that folder anywhere?

This is a fresh machine on which I've only installed Visual Studio 2017 Enterprise. What Nuget.Config should I check?

livarcocc commented 6 years ago

Check %appdata%/nuget/nuget.config.

Also @rohit21agrawal @nkolev92 is there a way to add diagnostics to see which nuget.config are being used by restore?

nkolev92 commented 6 years ago

Increase the verbosity level. Restore prints that at the end of each run.

Try

/v:d

CIPop commented 6 years ago

%appdata%/nuget/nuget.config

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
  </packageSources>
</configuration>
nkolev92 commented 6 years ago

@CIPop Can you do the following:

dotnet msbuild /t:GenerateRestoreGraphFile /p:RestoreGraphOutputPath=a.dg

and attach that a.dg file here?

Note that it will have your full dependency graph.

You can also look there for references of that folder.

dotnet msbuild /t:GenerateRestoreGraphFile /bl

and attaching the binlog would do the same trick as well.

CIPop commented 6 years ago

I did:

S:\cs (master -> origin)
λ nuget locals all -clear
Local resources cleared.

S:\cs (master -> origin)
λ git clean -xdf
λ build -verb d

I can't repro anymore even if I remove the workaround...

CIPop commented 6 years ago

Graph file: a.dg.txt

CIPop commented 6 years ago

@nkolev92 This isn't the first time I see this on a fresh Windows deployment. Somehow it went away after creating that folder and it doesn't seem to crash even if I deleted the workaround.

Closing for now - I can reopen when I rebuild my dev box again or I see this on a VM.

nkolev92 commented 6 years ago

Sounds good @CIPop

For reference, the dg spec file, has the evaluated FallbackFolders and the config file paths.

That's normally extremely helpful when investigating restore related issues.

CumpsD commented 6 years ago

I have the same error, but on Windows Subsystem Linux, Bash on Ubuntu on Windows

/usr/local/dotnet/sdk/2.1.101/Sdks/Microsoft.NET.Sdk/build/Microsoft.PackageDependencyResolution.targets(172,5): error MSB4018: NuGet.Packaging.Core.PackagingException: Unable to find fallback package folder 'C:\Program Files\dotnet\sdk\NuGetFallbackFolder'. [/mnt/d/code/bb-aiv/municipality-registry/src/MunicipalityRegistry.Api.Oslo/MunicipalityRegistry.Api.Oslo.csproj]

Which makes sense, since there is no C:\ either it is ~/.dotnet or /mnt/c/

nkolev92 commented 6 years ago

@CumpsD NuGet does not generate the fallback folder path, it's passed to it through a property.

I'd suggest opening an issue on CLI side.

To confirm that this is the case, you can generate the /v:d logs and upload them here for us to double check, or just look for the RestoreFallbackFolders property and it's value.

livarcocc commented 6 years ago

@CumpsD I would treat this as a different issue from this one. If this is consistently happening to you, please file a separate issue with repro steps and we can take a look.

@nkolev92 this is already the CLI repo 😄

CumpsD commented 6 years ago

I've "fixed" it by not relying on the fallback folder:

  <PropertyGroup>
    <RestoreFallbackFolders>clear</RestoreFallbackFolders>
  </PropertyGroup>
KirillOsenkov commented 6 years ago

I'm seeing this issue now. binlog.zip

@livarcocc can you glean anything from the attached .binlog?

livarcocc commented 6 years ago

KirillOsenkov in your case, the folder is being used as a NuGet source, it is just called Fallback folder. Note that the property being set is NuGetPackageFolders. I don't believe your issue has anything to do with this one.

nguerrera commented 6 years ago

We're talking on slack, but re-closing since I agree this is different. If there's an issue in CLI after chatting, we'll reopen a different one.

framebassman commented 5 years ago

I "fixed" it via remove dotnet clean before dotnet restore in my Dockerfile

VictorioBerra commented 4 years ago

If you are using Docker and getting this error, make sure you have a .dockerignore and do not copy over the bin and obj files.

**/bin
**/obj
**/out
**/.vscode
**/.vs
.dotnet
.Microsoft.DotNet.ImageBuilder

VIA: https://github.com/dotnet/dotnet-docker/blob/master/.dockerignore

tervoju commented 4 years ago

got "Unable to find fallback package folder 'C:\Program Files (x86)\Xamarin\NuGet\" awsome. When coding VS code Azure function. dotnet restore saved my day.

rob-ack commented 4 years ago

In my case i was using the Linux Subsystem (Ubuntu 18.04) and had Visual Studio 2019 still open in windows. This may block some file handles and can fail your build with this misleading error. Closing VS and performing clean and than build fixed it.

dsplank commented 4 years ago

I have been fighting with this error for days now and tried closing Visual Studio like @RobertAcksel suggested.... and now it works. If you get this error, make sure VS is not open with the projects you are trying to build/publish.

RobThree commented 4 years ago

I deleted some folder in the root of my C: drive (C:\Microsoft\Xamarin) after I got a new workstation and had (clean) installed Windows 10 & VS2019. I should've known... But the folder was empty (I guess? I'm quite sure I checked it and must've thought it was some leftover installer residue file, I don't make it a habit to delete folders actually containing files (and I have showing hidden files turned on)). Anyway:

Error occurred while restoring NuGet packages: The local source 'C:\Microsoft\Xamarin\NuGet\' doesn't exist.
1>------ Build started: Project: ConsoleApp1, Configuration: Debug Any CPU ------
1>C:\Program Files\dotnet\sdk\3.1.201\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(164,5): error MSB4018: The "GetPackageDirectory" task failed unexpectedly.
1>C:\Program Files\dotnet\sdk\3.1.201\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(164,5): error MSB4018: NuGet.Packaging.Core.PackagingException: Unable to find fallback package folder 'C:\Microsoft\Xamarin\NuGet\'.
1>C:\Program Files\dotnet\sdk\3.1.201\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(164,5): error MSB4018:    at NuGet.Packaging.FallbackPackagePathResolver..ctor(String userPackageFolder, IEnumerable`1 fallbackPackageFolders)
1>C:\Program Files\dotnet\sdk\3.1.201\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(164,5): error MSB4018:    at Microsoft.NET.Build.Tasks.NuGetPackageResolver.CreateResolver(IEnumerable`1 packageFolders)
1>C:\Program Files\dotnet\sdk\3.1.201\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(164,5): error MSB4018:    at Microsoft.NET.Build.Tasks.GetPackageDirectory.ExecuteCore()
1>C:\Program Files\dotnet\sdk\3.1.201\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(164,5): error MSB4018:    at Microsoft.NET.Build.Tasks.TaskBase.Execute()
1>C:\Program Files\dotnet\sdk\3.1.201\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(164,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
1>C:\Program Files\dotnet\sdk\3.1.201\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(164,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()
1>Done building project "ConsoleApp1.csproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

image

I did happen do have a WSL open (Debian, but I also have a Ubuntu WSL) but closing that and VS2019 didn't help; even after a dotnet clean:

image

Or dotnet restore:

image

My %appdata%/nuget/nuget.config contains no reference to "C:\Microsoft..." at all:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
  </packageSources>
  <packageRestore>
    <add key="enabled" value="True" />
    <add key="automatic" value="True" />
  </packageRestore>
  <bindingRedirects>
    <add key="skip" value="False" />
  </bindingRedirects>
  <packageManagement>
    <add key="format" value="0" />
    <add key="disabled" value="False" />
  </packageManagement>
</configuration>

@CumpsD suggestion here worked for me. But creating a new project results in the same error.

I don't want to have to add this to each and every project and I also don't want some "C:\Microsoft\Xamarin" folder in the root of my drive - stuff just doesn't belong there. And I sure as hell didn't put it there. Howver, simply creating the directory does help. For now.

I am now literally scanning my entire C: drive for any files containing C:\Microsoft\Xamarin as to find out where that setting could be buried...

s-bauer commented 4 years ago

I don't want to have to add this to each and every project and I also don't want some "C:\Microsoft\Xamarin" folder in the root of my drive - stuff just doesn't belong there. And I sure as hell didn't put it there.

Exactly! Who the hell had the idea to place the folder on the root drive?!

RobThree commented 4 years ago

At least I was able to find the file where the entry resides: C:\Program Files (x86)\NuGet\Config\Xamarin.Offline.config contains:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <fallbackPackageFolders>
    <add key="Xamarin Offline Packages" value="C:\Microsoft\Xamarin\NuGet\"/>
  </fallbackPackageFolders>
</configuration>

Renaming the file to Xamarin.Offline.config.bak (adding .bak) helped for me. No need for a C:\Microsoft\... directory in the root of my drive and no need to add <RestoreFallbackFolders>clear</RestoreFallbackFolders> to each and every project.

Now only to find out what (else) was in that C:\Microsoft\... directory and what else broke when I deleted it... I probably broke Xamarin, but I don't use that (for now) anyway. I just happened to check the box when installing VS2019 thinking "maybe, someday, why not already install it?"

yannduran commented 4 years ago

@RobThree I deleted that folder too, and of course ran into this problem. I deleted it because it was empty and I don't use Xamarin either.

I agree wholeheartedly that the folder doesn't belong in the root of the C: drive.

KUTlime commented 3 years ago

@nguerrera Hi there! You closed this but the problem remains. Would be better to hear the feedback and fix it?

radutomy commented 3 years ago

I deleted C:\Microsoft\Xamarin\NuGet\ thinking it wasn't important and it completely broke Xamarin. I tried some of the solutions above but nothing worked for me.

Thanks Microsoft for a great design choice - putting random folders in the root directory.

yannduran commented 3 years ago

@nguerrera (I'm assuming you work for Microsoft)

Is Microsoft going to do anything about moving this folder out of the root directory, other than keep closing this issue? Why are the people saying this is a problem being ignored? If there's a better issue for this to be discussed, please mention it here.

This is a ridiculous place to have this folder and people are going to keep deleting it, as it's usually empty and it's best practice to NOT have random folders in the root anyway. Microsoft doesn't care about consistency or that it's violating what it advises us to not do?

If it's intended to be a machine-wide folder, It should have gone into C:\Program Files (x86)\NuGet\Xamarin as the NuGet folder already exists. If it's intended to be a user folder, it should be somewhere in %localappdata%/nuget (the most obvious choice) or even somewhere in %appdata%/nuget. The one place it should NOT be in is in the root of the Windows drive.

nguerrera commented 3 years ago

I do work at Microsoft, but I have not worked on this project for a long time.

This Xamarin folder does not appear related to the SDK or the root cause of this closed bug. I am not sure how best to reach the Xamarin tooling owners, but I think that's who should look at this next. @KathleenDollard @marcpopMSFT can you help route this?

yannduran commented 3 years ago

@nguerrera

Thanks for your reply/help Nick!

KUTlime commented 3 years ago

Indeed, the easy solution here is to put the Xamarin fallback setting into %localappdata%/nuget and that is it. I can't believe it that we have to discuss it and complain about it for years now. 😒

unrealnerd commented 3 years ago

I was getting the same error when working with wsl1 and running dotnet clean

/usr/share/dotnet/sdk/5.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(241,5): error MSB4018: The "ResolvePackageAssets" task failed unexpectedly. [/mnt/c/DSC/src/flowable-poc/wf-ui/wf-ui.csproj]
/usr/share/dotnet/sdk/5.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(241,5): error MSB4018: NuGet.Packaging.Core.PackagingException: Unable to find fallback package folder 'C:\Program Files\dotnet\sdk\NuGetFallbackFolder'. [/mnt/c/DSC/src/flowable-poc/wf-ui/wf-ui.csproj]

running in this order worked!

dotnet restore
dotnet clean
marcpopMSFT commented 3 years ago

@Redth Some folks are finding that deleting an empty xamarin\nuget folder breaks things. Can you take a look and review if this issue likely goes away with the planned work for .NET 6? I would assume so but would want to double check.

Redth commented 3 years ago

I've asked around internally to see who owns this particular config file and offline cache. We'll get this over to the correct person to comment on why this is the folder used. Thanks.

yannduran commented 3 years ago

And please, for goodness sake, don't make us wait for .NET 6 for this to be addressed.

KUTlime commented 3 years ago

Totally agree with @yannduran. If we have to wait for a year for an easy-to-done fix, something is terribly wrong. A software by definition means a changeable product. If you can change it or at least easily, it isn't a software.

kamyker commented 3 years ago

2021 and I still have to close vs to build a project.

marcpopMSFT commented 3 years ago

The Xamarin specific NuGet issue is being tracked in this developer community ticket

ryepup commented 3 years ago

I ran into something similar; I had my codebase in WSL and had compiled it via once via Visual Studio 2019. After closing visual studio, I still got this error running dotnet build from inside WSL:

error MSB4018: NuGet.Packaging.Core.PackagingException: Unable to find fallback package folder 'C:\Program Files\dotnet\sdk\NuGetFallbackFolder'.

Variations on dotnet restore and dotnet clean didn't work, but this fixed it:

$ find -type d -iname bin | xargs rm -rf
$ find -type d -iname obj | xargs rm -rf

After that, dotnet build worked fine. I think my issue was mixing metadata between a windows-flavored compiler in Visual Studio with my linux-flavored compiler in WSL.

jomeno commented 3 years ago

I ran into this too and tried some of the suggestions in the comments. What solved it for me was removing this --no-restore flag from the RUN command in my Dockerfile.

This caused error RUN dotnet publish -c release -o /app --no-restore

This worked perfectly for me. RUN dotnet publish -c release -o /app

bganov commented 3 years ago

FWIW: This error was observed in VSfM (VS for Mac) 2019 in a multi-project solution. It manifested on only 2 projects - a supplemental project and a test project not related to the former project. Again, this is on Mac not Windows environment! I understand that most folks here are reporting and posting re:Windows, but I think adding my 2c to the thread will be beneficial to folks like myself (Mac users). Mainly because this thread is very high in the Google search results related to the issue.

So, without further ado -- VS has had long standing problem with 'Cleaning' a solution and contained projects. In my case there were only two offending projects, name and structure are irrelevant to the solution:

For each of the affected projects -- manually delete each one's "obj" and "bin" folders!

That did it for me!

djanri commented 2 years ago

manually delete each one's "obj"

Deleting "obj" folders was enough for me.

Adikkkkkkkk commented 1 year ago

thanks! the error faded away but which pitfalls in the work would i get with RestoreFallbackFolders mode :clear?

janseris commented 8 months ago

This still happens. For me, locally, with normal C# (no Xamarin), no docker, nothing. It is caused by outdated data in obj folder in one of transitively referenced projects which are not in solution. Why doesn't Clean in Visual Studio delete bin and obj folders??