Dotnet tool command doesn't respect nuget packages path

uladz-zubrycki commented 3 years ago

Description

It's possible to change the path for nuget packages globally by setting NUGET_PACKAGES environment variable (there're other means of achieving it check the documentation for details) . Once that done dotnet tool fails to start local tool. I guess it happens as tool was actually restored to nonstandard path, dotnet tool is not able to find it there and consistently asks to run dotnet tool restore.

Steps

Set NUGET_PACKAGES environment variable to some directory other than default %userprofile%/.nuget/packages
Initialize tool manifest by running dotnet new tool-manifest
Install some tool, e.g dotnet tool install paket
Try to run tool, e.g dotnet paket init

Expected result

Tool is invoked (which is paket init in current case)

Actual result

Error message saying Run "dotnet tool restore" to make the "paket" command available.

Running restore as suggested doesn't help. But everything works as expected after I manually move tool from the directory it was restored to to the %userprofile%/.nuget/packages.

Haven't checked whether the same is actual for global tool installation, but would guess it is.

Environment


.NET SDK (reflecting any global.json):
 Version:   5.0.101
 Commit:    d05174dc5a

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18363
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.101\

Host (useful for support):
  Version: 5.0.1
  Commit:  b02e13abab

.NET SDKs installed:
  3.1.404 [C:\Program Files\dotnet\sdk]
  5.0.101 [C:\Program Files\dotnet\sdk]

cfrob commented 3 years ago

There seems to be another quirk to this issue: After moving the locally installed tool to the default global-packages folder (as described by @v-zubritsky above), trying to restore the tool locally again using dotnet tool restore won't even install it locally anymore.

This aspect renders our current workaround for this issue useless: We simply called to tool by running dotnet giving it the full path to the installed tool-assembly, e.g. dotnet <packages_folder>/trx2junit/1.3.1/tools/netcoreapp3.0/any/trx2junit.dll This doesn't work reliably as the effect of dotnet tool restore depends on the state of the default global-packages folder.

wli3 commented 2 years ago

Cause: Local tools assuming there is only one nuget global cache, and it does not change (or it will start from scratch to populate the cache). It is also assumed by nuget. So local tools does not work well when changing the folder location on the fly.

manfred-brands commented 2 years ago

This seems to work for us. dotnet is a client/server type application. Make sure you shutdown the server with dotnet build-server shutdown after you change the environment variable. After that dotnet tool restore and dotnet tool run will use the new location.

rmarinho commented 2 years ago

I m also facing this issue, isn't no fix for this ? basically we need to just keep nuget packages on the default folder?

manfred-brands commented 2 years ago

I stand corrected. dotnet tool restore installs in the new place, but dotnet tool run doesn't look there.

brantburnett commented 10 months ago

I have also encountered this problem. I had to copy the restored tool from the custom path to the ~/.nuget/packages directory before dotnet tool run would work.

Note that I encountered this problem using the new Dev Drive feature in Windows, which recommends redirecting NUGET_PACKAGES to the Dev Drive for improved performance in the documentation. This seems like a significant gap in the new high performance development experience being touted.

https://learn.microsoft.com/en-us/windows/dev-drive/#storing-package-cache-on-dev-drive

xqiu commented 9 months ago

Same here, my C drive does not have any more spaces so that I have to move the nuget packages to another drive, then boom, dotnet ef stopped working! I have to use mklink /J to make a directory junction from %userprofile%/.nuget to my new directory to make it working.

MichalMucek commented 9 months ago

It's the same for me as in the issue description but with different tools. I have to handle it manually by moving the .NET tool packages to the service account's user folder.

kevkov commented 6 months ago

As a workaround the shim files in $HOME/.dotnet/toolResolverCache can be edited to fix the location of the tool. This way you don't need two nuget package dirs.

Akarinnnnn commented 5 months ago

Same here, my C drive does not have any more spaces so that I have to move the nuget packages to another drive, then boom, dotnet ef stopped working! I have to use mklink /J to make a directory junction from %userprofile%/.nuget to my new directory to make it working.

Make a junction from$env:USERPROFILE\\.nuget\\packages to $env:NUGET_PACKAGES also works.

baronfel commented 2 weeks ago

The fix we should go for here is to change the way we write and/or interpret the toolResolverCache entries for each tool. It may be helpful to look at a resolver cache entry in detail:

{
    "Version": "6.2.3",
    "TargetFramework": "net8.0",
    "RuntimeIdentifier": "any",
    "Name": "fantomas",
    "Runner": "dotnet",
    "PathToExecutable": "C:\\Users\\<username>\\.nuget\\packages\\fantomas\\6.2.3\\tools/net6.0/any/fantomas.dll"
}

This tool hardcodes the path at install-time. We should do one of the following:

write a relative path (i.e. within the package's directory) instead of an absolute path
ignore the PathToExecutable entirely for local tools and use the NuGet APIs at tool-run-time to compute the path to the dll to invoke
keep the current absolute path behavior, but align the behavior of dotnet tool restore and dotnet tool run so that a dotnet tool restore properly updates the absolute paths in the tool resolver cache and run is able to use that to look up the packages
remove the nuget package path from the PathToExecutable and write it as a new property, and then ensure that if the nuget package path changes that a tool restore operation is requested of the user to update the entries

My personal preference is option 1 or 2, but all 3 would solve the user-facing problem.

tudor-turcu commented 6 days ago

Any news about this? It would help a lot also the adoption of DevDrive: https://learn.microsoft.com/en-us/windows/dev-drive/

CaringDev commented 6 days ago

My personal preference is option 1 or 2, but all 3 would solve the user-facing problem.

As a user I'd prefer option 1, as that would be the least surprising / magic and help me debug potential issues. Second would be option 4

dotnet / sdk