search

fsprojects / FAKE

FAKE - F# Make
https://fake.build
Other
1.28k stars 581 forks source link

Reference assemblies are located in the wrong folder on Windows #2638

Closed nojaf closed 2 years ago

nojaf commented 2 years ago

Description

After upgrading to 5.21.0, I can no longer run my FAKE script on Windows.

There was a problem while setting up the environment:
-> Could not find referenced assemblies in path: 'C:\Users\nojaf\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0\ref\net6.0', please check installed SDK and runtime versions

Repro steps

dotnet fake -v build

C:\Users\nojaf\Projects\Datto\network-scanner>dotnet fake -v build
runOrBuild ({ Script = None
  ScriptArguments = []
  FsiArgLine = []
  Debug = false
  NoCache = false
  RestoreOnlyGroup = false
  VerboseLevel = Verbose
  IsBuild = true })
FAKE 5 - F# Make (5.21.0) (this line is written to standard error, see https://github.com/fsharp/FAKE/issues/2066)
prepareAndRunScriptRedirect(Script: C:\Users\nojaf\Projects\Datto\network-scanner\build.fsx, fsiOptions: "")
Writing 'C:\Users\nojaf\Projects\Datto\network-scanner\.fake\build.fsx\intellisense.fsx'
Restoring with paket...
The last restore is still up to date. Nothing left to do.
Retrieving the assemblies (rid: 'win10-x64')...
Using .Net 6 assemblies
Trying to resolve runtime version from network..
Calculating the runtime graph...
Loaded runtime json from: microsoft.netcore.platforms-5.0.3
resolved runtime version: 6.0.0
Resolved referenced SDK path: C:\Users\nojaf\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0\ref\net6.0
Performance:
 - Cli parsing: 132 milliseconds
 - Packages: 1 second
   - Creating Runtime Graph: 40 milliseconds
   - Retrieve Assembly List: 1 second
 - Runtime: 2 seconds
There was a problem while setting up the environment, see standard error for details.
There was a problem while setting up the environment:
-> Could not find referenced assemblies in path: 'C:\Users\nojaf\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0\ref\net6.0', please check installed SDK and runtime versions
   StackTrace:
        at Microsoft.FSharp.Core.PrintfModule.PrintFormatToStringThenFail@1439.Invoke(String message) in D:\workspace\_work\1\s\src\fsharp\FSharp.Core\printf.fs:line 1439
        at Fake.Runtime.FakeRuntime.retrieveInfosUncached@114(String cacheDir, Lazy`1 paketDependenciesFile, VerboseLevel logLevel, GroupName groupName, SdkAssemblyResolver sdkAssemblyResolver, FrameworkIdentifier framework, Rid rid, Rid ridNotVersionSpecific, Lazy`1 lockFile, Lazy`1 cache, Unit unitVar0) in D:\
a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 202
        at Fake.Runtime.FakeRuntime.getKnownDependencies@312-1.Invoke(Unit unitVar0) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 312
        at Fake.Runtime.CoreCache.getCached[a](FSharpFunc`2 getUncached, FSharpFunc`2 readFromCache, FSharpFunc`2 writeToCache, FSharpFunc`2 checkCacheUpToDate) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\CoreCache.fs:line 41
        at Fake.Runtime.FakeRuntime.getKnownDependencies@310(String cacheDir, Lazy`1 paketDependenciesFile, VerboseLevel logLevel, GroupName groupName, FileInfo lockFilePath, String dependencyCacheHashFile, String dependencyCacheFile, SdkAssemblyResolver sdkAssemblyResolver, FrameworkIdentifier framework, Rid ri
d, Rid ridNotVersionSpecific, Lazy`1 lockFile, Lazy`1 cache, Lazy`1 writeIntellisenseTask, FSharpFunc`2 readFromCache, FSharpFunc`2 writeToCache, Unit unitVar0) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 311
        at Fake.Runtime.FakeRuntime.knownDependencies@320.Invoke(Unit unitVar) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 320
        at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
        at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
        at System.Lazy`1.CreateValue()
        at System.Lazy`1.get_Value()
        at Fake.Runtime.FakeRuntime.paketCachingProvider(FakeConfig config, String cacheDir, Dependencies paketApi, Lazy`1 paketDependenciesFile, FSharpOption`1 group) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 323
        at Fake.Runtime.FakeRuntime.runScript(PrepareInfo preparedScript) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 579
        at Program.runOrBuild(RunArguments args) in D:\a\FAKE\FAKE\src\app\Fake.netcore\Program.fs:line 156

Expected behavior

FAKE script runs.

Actual behavior

The reference assemblies are being located from C:\Users\<user>\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref, on my machine they are stored on C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0.

Known workarounds

Copying the 6.0.0 folder from Program Files to AppData\Local solves the problem.

Related information

nojaf commented 2 years ago

Workaround

PS> $env:FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH = "C:\Program Files\dotnet" ; dotnet fake build

TheAngryByrd commented 2 years ago

Is code here seems to be the issue: https://github.com/fsprojects/FAKE/blob/9dac475110c5bf785cdcc28e2d4b327d8344a64c/src/app/Fake.Runtime/SdkAssemblyResolver.fs#L126-L161

Instead of setting dotnetHostPath once, it might be worth always trying all three potential paths set (ENV_VAR, User, then System), every time we go to execute something. Or maybe there's a better/safer lookup for this? cc @baronfel

@nojaf, I just want to confirm but I think you mentioned in slack that C:\Users\<user>\AppData\Local\Microsoft\dotnet\dotnet.exe did exist on your machine. Could you also show what other sdks are installed in the user path? (I'm more just curious here)

baronfel commented 2 years ago

I think the custom/user/system probing is slightly off there - the way to do that probing in a way that's consistent with the way other tooling expects is to

This is what we do in proj-info, and that's modeled off of the lookup logic in the .NET runtime/SDK itself

nojaf commented 2 years ago

@TheAngryByrd yes, "C:\Users\nojaf\AppData\Local\Microsoft\dotnet\dotnet.exe" does exists. I do have this folder as well: C:\Users\nojaf\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\3.1.0

SteveGilham commented 2 years ago

I'm not sure if this is some legacy install thing, as the files I have in the AppData\Local\Microsoft\dotnet tree are all from SDK 2.1, more than two years old, even after having uninstalled everything before 3.1 to tidy up after the net6.0 release.

Question -- is is safe just to delete the dotnet.exe from this (or even the whole tree)?

baronfel commented 2 years ago

if those directories aren't in your PATH and aren't shown when you run dotnet --list-sdks, then yes you can purge with abandon.

yazeedobaid commented 2 years ago

The Sdk resolver tries first to get user-installed locations before the default dotnet installation. So it seems that you have dotnet installed in a specific location and in default location. So things got mixed up. I think that net6 is installed in the default location but you have an installation that is in your AppData\Local directory. And resolver combined them as C:\Users\nojaf\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0\ref\net6.0

can you please provide output for dotnet --info

yazeedobaid commented 2 years ago

@TheAngryByrd yes, "C:\Users\nojaf\AppData\Local\Microsoft\dotnet\dotnet.exe" does exists. I do have this folder as well: C:\Users\nojaf\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\3.1.0

So yeah you have dotnet 3.1 in user specific location and net6 in default installation. And SDK resolver got mixed them.

I think we should try both; user-specific and the default location for resolved reference assemblies version. First, user-installed and then default.

So in your case it will be we try first if the path exists: C:\Users\nojaf\AppData\Local\Microsoft\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0\ref\net6.0 And if not (which is your case), we try C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.0

baronfel commented 2 years ago

I disagree, this logic is not aligned with how any other tooling works. it doesn't match how FSI typechecks the scripts, for example. the algorithm I linked above is the supported way to find the SDK directories and you really really should align with that.

yazeedobaid commented 2 years ago

@baronfel Thanks! If that is the correct algorithm and covers all the cases then we should definitely use it in FAKE. Does anyone want to help with this?

Thanks

baronfel commented 2 years ago

@TheAngryByrd said I should extract and publish this, but you can essentially take the code from ionide/proj-info wholesale to help here: https://github.com/ionide/proj-info/blob/main/src/Ionide.ProjInfo/Utils.fs#L7-L87

You can use the Paths.dotnetRoot lazy to get the path to the dotnet directory you should use as your dotnetHostPath in the code that @TheAngryByrd linked above, but then the rest of the code you have looks good to me.

nojaf commented 2 years ago

I want to take a stab at this, feels like an easy steal of Chet's code, so I'm game 😅.

mk185147 commented 2 years ago

I also see related issue. I have a custom dotnet version locally installed/copied by build pipeline and I start dotnet fake command with this version then I expect (and was legacy behavior) that DotNet.exec within fake script will execute this custom dotnet version. This was broken with those changes.

mattiasdrp commented 2 years ago

I guess my problem is the same as this one (I'm following the install instructions from here

❯ dotnet fake -v build
runOrBuild ({ Script = None
  ScriptArguments = []
  FsiArgLine = []
  Debug = false
  NoCache = false
  RestoreOnlyGroup = false
  VerboseLevel = Verbose
  IsBuild = true })
FAKE 5 - F# Make (5.21.0-alpha004) (this line is written to standard error, see https://github.com/fsharp/FAKE/issues/2066)
prepareAndRunScriptRedirect(Script: /home/mattias/fsharp/FsAutoComplete/build.fsx, fsiOptions: "")
Writing '/home/mattias/fsharp/FsAutoComplete/.fake/build.fsx/intellisense.fsx'
Restoring with paket...
The last restore is still up to date. Nothing left to do.
Retrieving the assemblies (rid: 'ubuntu.20.04-x64')...
Calculating the runtime graph...
Loaded runtime json from: microsoft.netcore.platforms-6.0.1
Performance:
 - Cli parsing: 223 milliseconds
 - Packages: 2 seconds
   - Creating Runtime Graph: 49 milliseconds
   - Retrieve Assembly List: 2 seconds
 - Runtime: 2 seconds
There was a problem while setting up the environment, see standard error for details.
There was a problem while setting up the environment:
-> Could not find referenced assemblies in path: '/usr/local/share/dotnet/packs/Microsoft.NETCore.App.Ref/6.0.0/ref/net6.0', please check installed SDK and runtime versions
   StackTrace:
        at Microsoft.FSharp.Core.PrintfModule.PrintFormatToStringThenFail@1439.Invoke(String message) in D:\workspace\_work\1\s\src\fsharp\FSharp.Core\printf.fs:line 1439
        at Fake.Runtime.FakeRuntime.retrieveInfosUncached@114(String cacheDir, Lazy`1 paketDependenciesFile, VerboseLevel logLevel, GroupName groupName, SdkAssemblyResolver sdkAssemblyResolver, FrameworkIdentifier framework, Rid rid, Rid ridNotVersionSpecific, Lazy`1 lockFile, Lazy`1 cache, Unit unitVar0) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 206
        at Fake.Runtime.FakeRuntime.getKnownDependencies@312-1.Invoke(Unit unitVar0) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 312
        at Fake.Runtime.CoreCache.getCached[a](FSharpFunc`2 getUncached, FSharpFunc`2 readFromCache, FSharpFunc`2 writeToCache, FSharpFunc`2 checkCacheUpToDate) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\CoreCache.fs:line 41
        at Fake.Runtime.FakeRuntime.getKnownDependencies@310(String cacheDir, Lazy`1 paketDependenciesFile, VerboseLevel logLevel, GroupName groupName, FileInfo lockFilePath, String dependencyCacheHashFile, String dependencyCacheFile, SdkAssemblyResolver sdkAssemblyResolver, FrameworkIdentifier framework, Rid rid, Rid ridNotVersionSpecific, Lazy`1 lockFile, Lazy`1 cache, Lazy`1 writeIntellisenseTask, FSharpFunc`2 readFromCache, FSharpFunc`2 writeToCache, Unit unitVar0) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 311
        at Fake.Runtime.FakeRuntime.knownDependencies@320.Invoke(Unit unitVar) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 320
        at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
        at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
        at System.Lazy`1.CreateValue()
        at System.Lazy`1.get_Value()
        at Fake.Runtime.FakeRuntime.paketCachingProvider(FakeConfig config, String cacheDir, Dependencies paketApi, Lazy`1 paketDependenciesFile, FSharpOption`1 group) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 323
        at Fake.Runtime.FakeRuntime.runScript(PrepareInfo preparedScript) in D:\a\FAKE\FAKE\src\app\Fake.Runtime\FakeRuntime.fs:line 579
        at Program.runOrBuild(RunArguments args) in D:\a\FAKE\FAKE\src\app\Fake.netcore\Program.fs:line 156
Hint: If you just upgraded the fake-runner you can try to remove the .fake directory and try again.

I'm just starting with dotnet so I have no idea what I should do to be able to install it

nojaf commented 2 years ago

I think you would need a slight variation of $env:FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH = "C:\Program Files\dotnet". Your dotnet will be located somewhere else on Unix but setting the FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH environment variable might work out for you.

yazeedobaid commented 2 years ago

Release 5.21.1 has been published, Thanks @nojaf

JohnTheGr8 commented 2 years ago

Unfortunately, I am still encountering this issue in Fake 5.21.1.

In my case, I am running Fake using the dotnet/sdk:6.0.101 docker image. (This all worked fine up to Fake 5.20.4)

The error I am getting is this:

There was a problem while setting up the environment: -> Could not find referenced assemblies in path: '/usr/bin/packs/Microsoft.NETCore.App.Ref/6.0.1/ref/net6.0', please check installed SDK and runtime versions

A workaround I've found, similar to what @nojaf initially recommended, is to set FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH to /usr/share/dotnet. It works with both Fake 5.21.0 and 5.21.1.

(please let me know if I should open a new issue)

mattiasdrp commented 2 years ago

@nojaf My issue is that my dotnet version is 6.0.101 so the issue is that there is no /usr/local/share/dotnet/packs/Microsoft.NETCore.App.Ref/6.0.0/ref/net6.0. I do have /usr/local/share/dotnet/packs/Microsoft.NETCore.App.Ref/6.0.1/ref/net6.0/ but I don't know how to tell FAKE to look for it.

I tried adding the proper path in FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH (with FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH="/usr/share/dotnet:$FAKE_SDK_RESOLVER_CUSTOM_DOTNET_PATH") and even ln -s /usr/share/dotnet /usr/local/share/dotnet but it doesn't work.

yazeedobaid commented 2 years ago

@JohnTheGr8 & @mattiasdrp could you please open new issues with your case to keep track of it. Thanks

mattiasdrp commented 2 years ago

@yazeedobaid Done in #2648