Open wli3 opened 5 years ago
From @natemcmaster on Friday, April 20, 2018 7:45:45 PM
Possible solution: the apphost should automatically default DOTNET_ROOT to the first dotnet.exe on PATH.
From @livarcocc on Friday, April 20, 2018 8:30:15 PM
This is by design and the decision we made on the host taking many different things in consideration. I believe the dotnet install message will guide you to setting DOTNET_ROOT if your dotnet is not installed in the default location.
From @natemcmaster on Friday, April 20, 2018 8:32:27 PM
My concern about this design is that the error message is not clear when there are some .NET Core runtime installations in C:\Program Files\dotnet, but not the right ones for the global tool. How would a user ever figure this out from this?
PS> dotnet-serve
The required library C:\Program Files\dotnet\host\fxr\2.0.7\hostfxr.dll does not support relative app dll paths.
From @livarcocc on Friday, April 20, 2018 8:33:14 PM
@steveharter is this something that we could improve in the host?
From @steveharter on Friday, April 20, 2018 9:42:35 PM
The error message could be extended to provide more information, but is really a short-term problem since once the hostfxr should be updated (once) it should be good forever for this particular error message.
Possible solution: the apphost should automatically default DOTNET_ROOT to the first dotnet.exe on PATH.
I'd prefer not to start probing %PATH%
From @natemcmaster on Friday, April 20, 2018 9:54:25 PM
I'd prefer not to start probing %PATH%
Can you elaborate? It seems like it would be a better user experience, but want to know why you would rather not.
I was expecting this PATH lookup behavior because it's one we depend on to make Visual Studio work SDKs installed into user-locations. It's also one less variable that we have to set to make CI builds work well. We already have a handful, and the list seems to be ever growing.
From @steveharter on Thursday, April 26, 2018 9:11:38 PM
@richlander
Probing %PATH% for dotnet.exe is not intuitive and somewhat error-prone as we can't exactly duplicate what the OS does. That's why we added %DOTNET_ROOT%
From @tmds on Friday, April 27, 2018 6:54:37 AM
I filed the same issue for Linux: https://github.com/dotnet/core-setup/issues/4078. I have the same expectation as @natemcmaster: dotnet
should be searched on PATH.
For source-build Red Hat, Fedora and CentOS, dotnet
is not installed at the 'default' location. So tools fail out-of-the-box π
CC @eerhardt @dleeapho @omajid
An alternative solution (https://github.com/dotnet/core-setup/issues/4078#issuecomment-384653329) is to allow source-build to specify the 'default' location. That would fix the source-build case, but not @natemcmaster's case. This would still require setting DOTNET_ROOT. Perhaps we can use this fallback order: DOTNET_ROOT, PATH probing, hard-coded path.
A pseudo Unix implementation for finding the root via PATH:
segments = split PATH by ':'
foreach segment
dotnet_path = concat(segment, "/dotnet")
if (stat(dotnet_path, out dotnet_stat) == SUCCESS && dotnet_stat.isExecutable)
if (realpath(dotnet_path, out dotnet_real) == SUCCESS)
return dirname(dotnet_real)
return null
From @dasMulli on Friday, April 27, 2018 8:55:52 AM
Aren't there other things already looking for dotnet.exe on the %PATH%? MSBuildSdkResolver AFAIK
This could lead to situations where dotnet-mytool
and dotnet mytool
may end up choosing a different runtime. The question here is what a user would expect..
From @tmds on Friday, April 27, 2018 9:02:17 AM
Independent of implementation, a user expects this to work:
$ dotnet tool install -g dotnet-serve
$ dotnet-serve
From @eerhardt on Friday, April 27, 2018 2:38:32 PM
The issue I have with probing the %PATH%
env var is that the %PATH%
variable is used to locate executables to run. In this case, we've already found the executable to run. The dotnet-serve
executable was located and loaded and started executing. The %PATH%
environment variable served its purpose.
Now, the dotnet-serve
executable needs to load some more libraries (ex. coreclr, System.Private.CoreLib, etc). Why should the PATH
variable be involved with how to locate more libraries? It's main purpose is for locating executables to be executed.
In my opinion it conflates the job/purpose of the PATH
environment variable. It can also lead to other side effects, such as having 2 "dotnet hives" installed on your machine. One on the PATH, and one not. Now if you invoke /path/to/dotnet tool install -g dotnet-serve
and then run dotnet-serve
it won't necessarily use the same "dotnet hive" you used to install it.
That's why we introduced a new environment variable that serves a single purpose - locating the "dotnet hive".
From @tmds on Friday, April 27, 2018 3:26:15 PM
@eerhardt what do you think about this order then: DOTNET_ROOT, hard-coded path, PATH probing.
From @livarcocc on Friday, April 27, 2018 4:40:15 PM
@steveharter @richlander
From @natemcmaster on Friday, April 27, 2018 5:02:46 PM
A agree with what @tmds said. It was my expectation that after dotnet tool install
, dotnet serve
would use the same dotnet used to install the tool. I think many users will expect this.
To @eerhardt's points:
Why should the PATH variable be involved with how to locate more libraries? It's main purpose is for locating executables to be executed.
I understand the engineering purity of this argument, but think there is another compelling point to consider. PATH probing is consistent with how the rest of the product behaves. Over the last few years, we have been teaching people that you can install .NET Core into a custom location. If you put it on PATH first, Visual Studio, VS Code, MSBuild.exe, and the command line will use that custom install location. The current behavior of global tools breaks from this pattern. And from what I'm hearing, the reasons for breaking from that pattern seem to be narrowly focused on what the apphost should do, not on the behavior of the .NET Core product as a whole.
Now if you invoke /path/to/dotnet tool install -g dotnet-serve and then run dotnet-serve it won't necessarily use the same "dotnet hive" you used to install it.
I agree with you here. I would not expect dotnet-serve to find /path/to/dotnet. That said, I'm not asking this. I'm asking for PATH probing.
Let me be completely clear why this is so maddening to me. The majority of .NET Core tools today are written so they can be invoked as dotnet <command>
(that's a subject for another conversation, but it's the reality of the current world.) If I have C:\custom\dotnet.exe
first on PATH, when invoking dotnet <command>
, the "dotnet" used is C:\custom\dotnet.exe
, but the CLI will launch <command>
part in a new context that ignores C:\custom\
. If nothing else, please set DOTNET_ROOT
from the CLI to the current location so any subprocesses launched by the CLI will resolve to the same dotnet.exe.
From @steveharter on Saturday, April 28, 2018 12:02:14 AM
Concerns
1) The OS does all of the %PATH% probing today, so this would be a new area for us to consider. We can't guarentee we follow the same rules as how the OS uses %PATH%
2) Performance: if there are 20 folders in %PATH% before we find a hit (or miss) we would be probing\reading 20 folders and possibly doing a stat
to ensure executable...
3) We would need to ensure this is not a security\hijack concern
That said, I agree that the scenario is real and proposed solution has merit. We did consider this during the design phase, I can PM you that link. However, that discussion for %PATH% was not considered as a last-resort fallback at the time; instead it was competing with %DOTNET_ROOT% as the primary mechanism.
From @tmds on Saturday, April 28, 2018 6:52:17 AM
The OS does all of the %PATH% probing today
On Linux this is implemented in user-space. For example, corefx implementation is here: https://github.com/dotnet/corefx/blob/9bfa3bd511a6a2152de8192332b0e439917e65e0/src/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs#L480-L498
Performance: if there are 20 folders in %PATH% before we find a hit (or miss) we would be probing\reading 20 folders and possibly doing a stat to ensure executable...
A lot of things get probed (executable on PATH, libraries on LD_LIBRARY_PATH, ...) and we don't perceive slowness. The earlier on PATH, the sooner the match.
We would need to ensure this is not a security\hijack concern
Using data from PATH is as secure as using data from DOTNET_ROOT.
that discussion for %PATH% was not considered as a last-resort fallback at the time
For the reasons explained by @natemcmaster my preferred order is _DOTNETROOT, PATH probing, hard-coded path
For source-build, we want to be able to set the hard-coded path. If path probing is implemented, we can rely on that for now. If there is no path probing, we need to make the hard-coded path a setting that can be specified from source-build.
From @enricosada on Wednesday, May 2, 2018 4:08:19 PM
same issue with 2.1.300-preview2-008530
on win-10
i installed that from zip and added to PATH (that's what we do in build servers, no global installs)
if i do:
dotnet install dotnet-mytool --tool-path my\tools\dir
the install is ok
but doesnt works running it directly with my\tools\dir\dotnet-mytool.exe
giving error:
The required library C:\Program Files\dotnet\host\fxr\2.0.6\hostfxr.dll does not support startupconfig.json functionality.
As a note, that installed is higher priority in PATH
(where dotnet
and dotnet --info
use that) than the .net sdk installed globally, but seems to resolve the globally installed dotnet sdk. global.json
doesnt help.
setting DOTNET_ROOT
env var to the unzip path fix the issue but is a lot less nice.
From @tmds on Thursday, May 3, 2018 6:14:56 AM
@richlander @KathleenDollard can you chime in please? It would be best to decide how this works for 2.1.
From @richlander on Thursday, May 3, 2018 4:44:00 PM
This is a real issue. We discussed it when we were designing global tools. We didn't discuss the point that @natemcmaster raised on installation and execution of a tool not agreeing on .NET Core location. That's a real challenge.
I'm not going to propose anything just yet, but need to information collect a bit more.
Aren't there other things already looking for dotnet.exe on the %PATH%? MSBuildSdkResolver AFAIK
Can someone point me to the code that does this?
From @natemcmaster on Thursday, May 3, 2018 4:55:39 PM
From @enricosada on Thursday, May 3, 2018 6:17:21 PM
as a note.
2.1.300-preview2
from unzippingPATH
in front (=> priority), like set PATH=%CD%;%PATH%
dotnet tool install dotnet-serve --tool-path mybin
(install to specific dir)You can invoke the tool using the following command: dotnet-serve
Tool 'dotnet-serve' (version '0.3.0') was successfully installed.
But both are broken, also direct run of the .exe:
E:\>dotnet serve
The required library C:\Program Files\dotnet\host\fxr\2.0.6\hostfxr.dll does not support startupconfig.json functionality.
E:\>mybin\dotnet-serve.exe
The required library C:\Program Files\dotnet\host\fxr\2.0.6\hostfxr.dll does not support startupconfig.json functionality.
at least the second should work. seetting DOTNET_ROOT
fix that.
the dotnet
is found on PATH
correctly
E:\>where dotnet
c:\dotnetcli\dotnet-sdk-2.1.300-preview2-008530-win-x64\dotnet.exe
C:\Program Files\dotnet\dotnet.exe
but the host\fxr\2.1.0-preview2-26406-04\hostfxr.dll
is not probed, so falling back to previous globally installed C:\Program Files\dotnet\host\fxr\2.0.6
From @tmds on Tuesday, May 15, 2018 9:35:59 AM
@richlander do you have an update?
From @dasMulli on Tuesday, May 15, 2018 10:55:06 AM
One of the key side-by-side promises of .NET Core was that you could just run an executable from a folder and add things to the PATH, it would be very nice if global tools would follow this approach.
If the PATH was probed as a last resort fallback then it may not be a breaking change after 2.1.300 and could maybe be done for 2.1.400? It would only enable things that would not have worked before anyway and wouldn't break things that used to work before.
From @livarcocc on Tuesday, May 15, 2018 5:06:06 PM
We are working on trying to find alternatives to improve this scenario. I will post back once we know more.
From @tmds on Thursday, May 31, 2018 7:35:54 AM
I'm comparing the gobal tools with npm tools. observations:
-g
for npm is independent of users, while -g
for dotnet installs into a user folder.npm
use node
that is on PATH
@livarcocc @richlander if you are changing the design of this, please consider doing a https://github.com/dotnet/designs.
From @mattwelke on Sunday, July 1, 2018 1:32:29 AM
I seem to be running into this after upgrading to 2.1 on Arch Linux and attempting to use EF Core migrations. Because migrations are now implemented as a global tool, I receive a message instructing me to install it:
$ dotnet ef migrations add Init
Cannot find command 'dotnet ef', please run the following command to install
dotnet tool install --global dotnet-ef
But when trying to install it, I get this message. I tried to set DOTNET_ROOT
to the executable my system's package manager set up, but it still doesn't work. I also tried setting DOTNET_ROOT
as the directory where that executable lies. It still didn't help. What should DOTNET_ROOT
be set as?
$ echo $DOTNET_ROOT
/usr/bin/dotnet
$ which dotnet
/usr/bin/dotnet
$ dotnet ef migrations add Init
A fatal error occurred, the required library libhostfxr.so could not be found.
If this is a self-contained application, that library should exist in [/home/mwelke/.dotnet/tools/.store/dotnet-ef/2.1.1/dotnet-ef/2.1.1/tools/netcoreapp2.1/any/].
If this is a framework-dependent application, install the runtime in the default location [/usr/share/dotnet] or use the DOTNET_ROOT environment variable to specify the runtime location.
$ export DOTNET_ROOT=/usr/bin
$ dotnet ef migrations add Init
A fatal error occurred, the required library libhostfxr.so could not be found.
If this is a self-contained application, that library should exist in [/home/mwelke/.dotnet/tools/.store/dotnet-ef/2.1.1/dotnet-ef/2.1.1/tools/netcoreapp2.1/any/].
If this is a framework-dependent application, install the runtime in the default location [/usr/share/dotnet] or use the DOTNET_ROOT environment variable to specify the runtime location.
$ echo $DOTNET_ROOT
/usr/bin
From @wli3 on Sunday, July 1, 2018 10:38:07 PM
@welkie the layout under DOTNET_ROOT should be like this. The directory that has the dotnet executable but not the symbol link
.
βββ dotnet
βββ host
βΒ Β βββ fxr
βββ LICENSE.txt
βββ sdk
βΒ Β βββ 2.1.300-rc1-008673
βΒ Β βββ NuGetFallbackFolder
βββ shared
βΒ Β βββ Microsoft.AspNetCore.All
βΒ Β βββ Microsoft.AspNetCore.App
βΒ Β βββ Microsoft.NETCore.App
βββ ThirdPartyNotices.txt
From @tmds on Monday, July 2, 2018 4:39:52 AM
This finds dotnet on PATH and determines DOTNET_ROOT based on that:
export DOTNET_ROOT=$(dirname $(realpath $(which dotnet)))
From @mattwelke on Monday, July 2, 2018 8:36:41 PM
@wli3 @tmds Thank you both for your help. I understand now where the env var is supposed to point and the shortcut command to locate it based on the path of the executable worked perfectly for me. I just added that to my .bashrc
file.
From @poke on Saturday, July 7, 2018 8:39:08 AM
Python solves this problem by putting tool executables into a folder that is inside the Python installation directory. So every installation can have its own set or tools.
Wouldn't it possible to do this as well for custom Core installations? E.g. put the tools into folder inside the folder that has the dotnet executable. Tools would then look up and check if there's a dotnet that they can use and otherwise fall back to a global path.
From @dasMulli on Saturday, July 7, 2018 9:30:19 AM
The problem here is that on most systems, dotnet
will be installed somewhere that would require administrative access (windows admin, superuser permission on *nix) to create directories in. It would also lead to users sharing their tools should they be installed in a global directory.
From @poke on Saturday, July 7, 2018 10:57:23 AM
But if I use a custom path, isnβt it that I would usually want to restrict the program to only that path?
When I think about my shared Linux days where I was trapped as a normal user to my own home directory, whenever I installed something, I would install it in my home directory, the only place I do have write access to. I think if I install .NET Core into e.g. /home/poke/dotnet-core/
, then I would also expect global dotnet tools to be installed in there.
From @tmds on Wednesday, July 11, 2018 12:39:51 PM
Python solves this problem by putting tool executables into a folder that is inside the Python installation directory. So every installation can have its own set or tools.
npm is similar. When installing with -g, that becomes part of the npm installation.
The problem here is that on most systems, dotnet will be installed somewhere that would require administrative access (windows admin, superuser permission on *nix) to create directories in.
npm global means the tool becomes available for everyone using that npm install. So if it is the system wide installation, it makes sense the install requires elevated permissions.
dotnet global means for this user. (I think this was named global because DotNetCliToolReference is per-project.)
From @KathleenDollard on Wednesday, July 11, 2018 2:48:30 PM
@tmds We struggled with using --global
since it is user, not machine. But we felt that the majority of machines with the SDK installed would be single user machines, and thus we felt that the majority of people would never notice and the rest would think "oh, sort of global". We definitely wanted to avoid requiring admin to install global tools.
This problem, being able to find the runtime for the global tool is clearly one we should solve.
From @dasMulli on Wednesday, July 11, 2018 3:29:46 PM
I like the "user-global" approach better since it also works great on shared machines like in educational facilities (the only bug hurting there is with global /tmp restrictions - https://stackoverflow.com/questions/45950644/prevent-net-core-2-0-from-leaving-files-in-tmp-on-rhel7 / https://github.com/Microsoft/msbuild/issues/1479).
A fallback lookup for dotnet
on the PATH
is also similar to all the scripts starting with shebangs like
#!/usr/bin/env node
#!/usr/bin/env python3
From @tmds on Wednesday, July 11, 2018 3:56:06 PM
I don't consider one better than the other. I think of them as different scopes for different use-cases. I wish --global
was named --user
(... naming is hard).
From @tmds on Thursday, December 6, 2018 3:32:53 PM
The framework dependent apphost (https://github.com/dotnet/sdk/pull/2578) is also affected by this.
From @AArnott on Sunday, December 9, 2018 6:27:46 PM
Installing a local tool and running it passes on Linux-based microsoft/dotnet
docker images but fails on Hosted OSX agents from Azure Pipelines.
From @jnm2 on Tuesday, May 21, 2019 2:16:23 AM
I'm still getting this error on Azure Pipelines macOS even with DOTNET_ROOT set:
- bash: |
export DOTNET_ROOT="$(dirname "$(command -v dotnet)")"
echo "DOTNET_ROOT is: $DOTNET_ROOT"
which dotnet
~/.dotnet/tools/dotnet-cake --target=Test --test-run-name=macOS --configuration=Release
Result:
DOTNET_ROOT is: /usr/local/bin
/usr/local/bin/dotnet
A fatal error occurred, the required library libhostfxr.dylib could not be found.
If this is a self-contained application, that library should exist in [/Users/vsts/.dotnet/tools/.store/cake.tool/0.33.0/cake.tool/0.33.0/tools/netcoreapp2.1/any/].
If this is a framework-dependent application, install the runtime in the default location [/usr/local/share/dotnet] or use the DOTNET_ROOT environment variable to specify the runtime location.
What options are there?
From @natemcmaster on Tuesday, May 21, 2019 3:30:26 AM
Try changing this line to include readlink
, which follow the symlink to the actual location of dotnet.
export DOTNET_ROOT="$(dirname "$(readlink "$(command -v dotnet)")")"
From @jnm2 on Sunday, May 26, 2019 8:28:57 PM
@natemcmaster Thank you! That works.
From @trondhindenes on Tuesday, July 30, 2019 5:50:58 PM
as a user who just supports dev teams I can't express how much confusion and frustration dotnet CLI causes. There's so much mess. We use the dotnet install scripts (https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-install-script) on Linux and making this all work with multiple frameworks/cli versions is just a mess. Please improve so we don't have to waste time getting seemingly simple things like "dotnet tool" to work.
From @baracoder on Wednesday, August 14, 2019 9:49:48 AM
strings .dotnet/tools/dotnetsay|grep dotnet
/usr/share/dotnet
.store/dotnetsay/2.1.4/dotnetsay/2.1.4/tools/netcoreapp2.1/any/dotnetsay.dll
When the shim is created, the path for the target dll is set in the apphost template. In the same step the string for dotnet-root could be set. Would this be an option?
From @grantzvolsky on Thursday, August 22, 2019 1:17:31 PM
For those using the snap package, the libhostfxr.so
error can be fixed as follows.
export DOTNET_ROOT=/snap/dotnet-sdk/current
Complete configuration for Ubuntu development: /etc/profile.d/04-dotnet-cli-tools.sh
#!/bin/bash
export DOTNET_ROOT=/snap/dotnet-sdk/current
export MSBuildSDKsPath=$DOTNET_ROOT/sdk/$(${DOTNET_ROOT}/dotnet --version)/Sdks
export PATH="${PATH}:${DOTNET_ROOT}"
export PATH="$PATH:$HOME/.dotnet/tools"
From @KallDrexx on Thursday, October 17, 2019 1:33:55 PM
I just want to add an anecdote I encountered that caused me a huge amount of confusion, and no amount of dotnet install
docs helped because I never ran dotnet install
.
I work at a bank and we don't have admin access to our machines. To not have to deal with the pain that it is getting someone to install .net core 3.0 for me I downloaded the binaries, placed them in c:\users\<name>\dotnet
, and added that to my path. I have been successfully doing .net core 2.2 and 3.0 development that way for a while (due to Rider not needing admin to install).
However, I downloaded Linqpad6 (which is a .net core 3.0 application) and it would not run. Running the executable would do absolutely nothing, and trying to dotnet Linqpad6.exe
would give an error about hostpolicy.dll
not being found (and no mention of any environment variables).
After a back and forth with the Linqpad dev I ended up doing a really round about testing that finally gave me the error above about the DOTNET_ROOT
environment variable not being set. Setting this fixed all my issues.
So my point is, this has the potential to really cause confusion for downloaded .net core precompiled binaries and it's actually very hard to find out that the solution is DOTNET_ROOT
.
At a minimum I believe that if DOTNET_ROOT
is not set then all executions of dotnet
cli should give a warning about this not being set.
This issue also happens on the default Visual Studio online vm. dotnet tool install -g fake-cli
does not add fake as a command that can be run even though it says everything installed correctly. Trying to run ~/.dotnet/tools/fake
causes fake to fail with A fatal error occured, the required library libhostfxr.so could not be found.
I'm using a surface pro x so since there is no ARM64 build available for vscode I thought I could get it running on visual studio online, and it actually did work very well until I hit this issue. Not really sure what to try next. Yes there is emulation for 32 bit but the visual lag is insufferable even for a regular remote desktop worker like me. I guess I'll keep remoting into my desktop pc for now...
@voronoipotato I've been using the following successfully for VS Online:
export DOTNET_ROOT=$(dirname $(realpath $(which dotnet)))
export PATH="$PATH:$HOME/.dotnet/tools"
@voronoipotato Just wanted to give an update and let you know that this issue has been fixed for new codespaces!
Awesome! Thank you so much for the heads up.
@wli3 The libhostfxr.so issue has been plaguing me the whole day, so I wanted to express thanks for documenting that tip. It should really be documented more prominently :man_facepalming:
From @natemcmaster on Friday, April 20, 2018 7:07:14 PM
Steps to reproduce
Download .NET Core CLI to $env:USERPROFILE/.dotnet/x64 and install a global tool
Expected behavior
The generated shim dotnet-serve.exe should find and use the dotnet on PATH to launch the global tool.
Actual behavior
If no .NET Core is installed in
C:\Program Files\dotnet
If .NET Core is installed in
C:\Program Files\dotnet
, but doesn't have the right shared framework versionEnvironment data
dotnet --info
output:Copied from original issue: dotnet/cli#9114