Write guidance on how to launch (non apphost) apps

[richlander]

There are scenarios where apps need to be launched with dotnet host. The big question is how to find the host, with the two obvious places to look being:

• PATH
• apphost configuration (DOTNET_ROOT, /etc/dotnet/install_location, ...)

I don't believe we've documented this.

This question may be similar to how one locates node from code given the use of nvm. I haven't looked at that pattern/guidance, but it seems similar.

[ghost]

Tagging subscribers to this area: @vitek-karas, @agocke, @vsadov See info in area-owners.md if you want to be subscribed.

Issue Details

---

There are scenarios where apps need to be launched with `dotnet` host. The big question is how to find the host, with the two obvious places to look being: - `PATH` - apphost configuration (`DOTNET_ROOT`, `/etc/dotnet/install_location`, ...) I don't believe we've documented this. This question may be similar to how one locates `node` from code given the use of `nvm`. I haven't looked at that pattern/guidance, but it seems similar.

Author:	richlander
Assignees:	-
Labels:	`area-Host`
Milestone:	-

[vitek-karas]

Depends where you want to do this. In a script something like which dotnet is probably a good answer. From code you could use nethost library, it finds hostfxr for a given application and from there you can go up 2 levels where dotnet should live. nethost will look at the same things as apphost (if used correctly): https://github.com/dotnet/runtime/blob/main/docs/design/features/native-hosting.md#new-host-binary-for-finding-hostfxr

[jaredpar]

This comes up frequently in the C# compiler source code as there are a number of cases where we need to launch a managed process. Effectively we have csc.dll and need to launch it as a process but that requires us first finding a dotnet to use. The pattern for finding dotnet hasn't been documented and as a result our implementation has been derived by looking at ENV variables set by the runtime, reviewing patterns in other parts of the .NET SDK, etc ... Basically trying to draw from other examples.

There are generally two scenarios I think about in this area:

1. How the C# compiler is executed in the .NET SDK. Essentially what should be the experience when the customer types dotnet build?
2. How the C# compiler is executed within our own infrastructure. There is a strong incentive for our own infra to be able to control the dotnet that is used when running tools in the SDK. For example the runtime repo uses this to effectively dogfood the latest runtime against SDK tools. This is very valuable.

After thinking this through this morning I think the right approach going forward is to only consult $PATH. Basically find the first dotnet instance and use that. This mostly closely mirrors how a customer would run our own tooling and is most likely to line up with their expectations. Basically dotnet build uses the same host as dotnet exec csc. Further I think this should apply to any tool in the SDK, not just the compiler.

That would have the following impact on our product and testing:

1. Our product code which today consults $DOTNET_ROOT or $DOTNET_HOST_PATH for the purpose of finding dotnet should cease doing so. Instead consult $PATH only.
2. Our infra code which was trying to insert a dotnet for testing / dogfood purposes by changing values like $DOTNET_ROOT should cease doing so. Instead that specific infra run should prepend the directory that contains the recently built dotnet to $PATH

[jaredpar]

If you're asking what is $DOTNET_HOST_PATH I had the same question. In researching through our code I believe that it is effectively an arcade invention. It's a mechanism for us to control where arcade puts dotnet into our enlistments. Unfortunately this seems to have filtered into our actual production code as well. That feels wrong.

Perhaps I have the history / intent of this variable wrong and if so please correct me 😄

[vitek-karas]

I think DOTNET_HOST_PATH is something SDK uses - there are many use cases in that repo: https://github.com/search?q=repo%3Adotnet%2Fsdk%20DOTNET_HOST_PATH&type=code I think it uses it to pass the location of the host from the parent to the child processes, so that child processes use the same SDK/runtime install which was used to run the top level command. It's possible Arcade uses it as well. But I could be wrong. For the usage inside SDK though we might want to keep looking at DOTNET_HOST_PATH. The problematic scenario is for example:

/my/private/install/dotnet build

This should spawn the child nodes for msbuild, csc and so on using the runtime/SDK from /my/private/install and not look for it on the PATH. For example if I have only installed 7.0 globally (so it's on PATH) but then I run /my/private/8.0-preview.7/dotnet then things should just work.

Unless we also change SDK to modify PATH for its child processes.

Outside of SDK I agree that PATH is probably the best approach. Installers, scripts and users will set PATH to point to dotnet.exe because we tell them to and because they need it anyway (so that they can run dotnet build and similar). It will also be set in CI systems to point to the dotnet from the SDK they want used and so on. DOTNET_ROOT in itself is not enough and there's much more involved algorithm how to find the runtime if that variable is not set. That's basically what nethost implements.

[jaredpar]

I think DOTNET_HOST_PATH is something SDK uses -

I 100% agree it uses it, I'm questioning why it does that. The $DOTNET_HOST_PATH does not appear to be a variable that we support as a product. The only places I can find where it is set is through arcade, not the customer facing product. As such I'm questioning whether or not those uses in the SDK are correct and at the moment I don't believe they are.

For the usage inside SDK though we might want to keep looking at DOTNET_HOST_PATH.

Why though? I'm struggling to see what value it provides vs. always using $PATH and just modifying that to use it.

If we do want a variable that special cases the SDK though then

1. We need to universally use it. Today it's hit and miss from what I can see.
2. Having it called DOTNET_ feels wrong. This is for effectively our internal testing purposes. Giving it a name like DOTNTE_ feels like we're elevating it to an actual .NET feature.

[vitek-karas]

For the usage inside SDK though we might want to keep looking at DOTNET_HOST_PATH.

Why though? I'm struggling to see what value it provides vs. always using $PATH and just modifying that to use it.

I realized this as well after I sent the response. It would be simpler if SDK just modified PATH for its child processes I assume and then everything could use `PATH.

Unfortunately PATH is not great to use - especially on Windows. Historically our guidance (given to us by other teams) was to try to not parse PATH and make sense out of it. But maybe it's OK for just spawning processes, since we can let the OS do it for us I assume.

[jaredpar]

Historically our guidance (given to us by other teams) was to try to not parse PATH and make sense out of it

Yeah I'm not really thrilled about parsing $PATH either. Ended up just stealing the code from msbuild that does it. This is essentially how our entire build tool stack works today so it seems like a good source to draw some code from.

Would be nice to have an API for FindOnPath but that seems unlikely. Also I'd still have to polyfill that for downlevel targets.

@rainersigwald, @baronfel: have you all ever considered exposing a FindToolPath on ToolTask or MSBuild APIs in general that we could use to find dotnet? That way the logic for parsing $PATH is centralized.

[rainersigwald]

It's come up before but I couldn't find an issue so https://github.com/dotnet/msbuild/issues/9018.

[agocke]

@vitek-karas is there a managed API for finding the location of the muxer? Is this something we want to provide?

[vitek-karas]

No - we don't have an API like that. There's something similar in SDK: https://github.com/dotnet/sdk/blob/a4b77d4fd5e878fd42aef22d86c0668c90888da6/src/Resolvers/Microsoft.DotNet.MSBuildSdkResolver/MSBuildSdkResolver.cs. And there seem to be other versions of this elsewhere, for example: https://github.com/microsoft/MSBuildLocator/blob/master/src/MSBuildLocator/DotNetSdkLocationHelper.cs (this one even mentions that maybe it should use nethost but doesn't). dotnet test also has something similar, where it is actually looking for dotnet.exe in some cases.

I know we've had several discussions with various SDK teams about this in the past, but no specific result. The problem is that there are like 5 versions of this code each with its own set of quirks and it's too late to change that.

[jaredpar]

There is one other view to consider which was brought up in https://github.com/dotnet/roslyn/issues/69023. Essentially all the tools that execute as part of an SDK action should use the same dotnet host.

For example:

> export PATH=/some/dotnet/install:$PATH
> which dotnet
/some/dotnet/install/dotnet
> /some/dotnet2/install/dotnet build 

In this view point tools like the compiler would be launched using /some/dotnet2/install/dotnet even though it is not the first dotnet on $PATH. Or possibly it's not on $PATH at all.

I'm certainly sympathetic to this point of view. Mostly because the opposite view seems quite strange. Basically if I launch msbuild with one dotnet host and then suddenly other tools are launching with another dotnet host that is effectively revealing implementation details of the build. Most of the time they are the same cause the primary customer use case is dotnet build without explicit qualification of which dotnet.

If there was an easy way from a process to find the dotnet host that created it then I'd lean towards this as the primary mechanism for finding the dotnet host. That cannot be the sole mechanism for finding dotnet though as we still need to fall back to $PATH lookup in cases where our .NET core tools are loaded from .NET Framework processes.

[agocke]

I think an API called something like AppContext.ExecutingMuxerPath makes sense -- the idea is that we would internally use the native hosting APIs to grab the location of hostfxr, and then from there we would try to find dotnet.exe (the muxer) in the expected location.

The problem being that there are a number of uncommon configurations where this will not work, including self-contained deployments where there just isn't a muxer to find. So this API would probably return null in a variety of cases where it's not possible to return a useful path.

@jaredpar Would this address all scenarios you're thinking of? I think the basic idea is that this would fix the problem, "I would like to start a new dotnet process using the same runtime/host that I'm running under." It wouldn't try to solve the problem of launching the "first" .NET process (either from native code or from the desktop framework). In those cases you would have to rely on PATH being appropriately configured, or use the native hosting APIs.

Another limitation would be that the muxer serves double-duty as the entry point to both the runtime and to the SDK. There's no guarantee that an SDK will actually be present in any given dotnet process though. So people have to be careful to only use this to execute DLLs, not run SDK commands.

[xoofx]

For the usage inside SDK though we might want to keep looking at DOTNET_HOST_PATH.

Why though? I'm struggling to see what value it provides vs. always using $PATH and just modifying that to use it.

I'm also not a fan of overriding PATH and looking into it, because it means that we can't use a dotnet exe without wrapping it with a batch/pwsh to setup this variable. That's really not great.

Even if DOTNET_HOST_PATH is not official, it seems that it is still considered and used all over the places as the primary location to look before going to the path, so why not making it official? Could we then always enforce it from dotnet.exe? (when run for the first time without a DOTNET_HOST_PATH?)

I think an API called something like AppContext.ExecutingMuxerPath makes sense -- the idea is that we would internally use the native hosting APIs to grab the location of hostfxr, and then from there we would try to find dotnet.exe (the muxer) in the expected location.

Having an API would be nice, but I would think that making DOTNET_HOST_PATH first class would be simpler and would even make it working with launching intermediate sub-child process shells that inherit from this variable and can resolve to dotnet.exe without having to explicitly propagate this variable around. It makes it more consistent to stick with an environment variable to propagate such things between managed and native processes.

[jaredpar]

"I would like to start a new dotnet process using the same runtime/host that I'm running under.

For me at least that is the primary problem to solve. Basically I have a suite of tools that launch each other and I want them to use the same host.

[jaredpar]

Having an API would be nice, but I would think that making DOTNET_HOST_PATH first class would be simpler

This is a proven approach we've used in MsBuild and it functions well for this type of scenario. Perhaps this new API just seeds this value more automatically.

[agocke]

@elinor-fung Any thoughts on the above? Either a managed API, or setting DOTNET_HOST_PATH environment variable when initializing the runtime?

[elinor-fung]

setting DOTNET_HOST_PATH environment variable when initializing the runtime

I don't really want to get the host itself into the business of setting environment variables. I think it would be reasonable to enable something like AppContext.GetData("DOTNET_HOST_PATH").

"I would like to start a new dotnet process using the same runtime/host that I'm running under.

For me at least that is the primary problem to solve. Basically I have a suite of tools that launch each other and I want them to use the same host.

Is it the same host that is desired, or the same runtime? Given https://github.com/dotnet/designs/pull/303, those could be different locations. My impression is that the same runtime would be the desire.

[vitek-karas]

What about versions? Even if we provide API to get to the same runtime location, there might be several runtimes (and SDKs) installed there. Are the scenarios such that we would also want to be able to use the same version of the runtime/SDK as the currently running process? (I must admit I don't know exactly how we would do that, just asking what the scenario calls for).

[rainersigwald]

For build scenarios I think we want "same runtime and SDK as current MSBuild process"--at least for SDK built-in tooling like Roslyn. Tools packaged in NuGet packages may want more flexibility but that feels like a good default.

[xoofx]

I don't really want to get the host itself into the business of setting environment variables

On that topic, I have been having a difficult time to track down which process/main is actually setting DOTNET_HOST_PATH? When we are referring to DOTNET_HOST_PATH, is it acknowledged that it is only about the muxer? (dotnet exe, in the SDK situation)

[xoofx]

For build scenarios I think we want "same runtime and SDK as current MSBuild process"--at least for SDK built-in tooling like Roslyn. Tools packaged in NuGet packages may want more flexibility but that feels like a good default.

By curiosity, how do we propagate from process to process the same runtime? (assuming we go through the dotnet muxer?) Do we point to the dotnet path/sdk/x.y.z/dotnet.dll in a specific SDK version folder?

[jaredpar]

I don't really want to get the host itself into the business of setting environment variables. I think it would be reasonable to enable something like AppContext.GetData("DOTNET_HOST_PATH").

I understand, and agree with, the reluctance for the runtime to be setting environment variables. At the same time though it would seem odd if the runtime is giving an API to find the host executable but then our SDK tool chain is effectively inventing another mechanism via $DOTNET_HOST_PATH environment variable. If we go with a new API then I think we need to consider moving the SDK to using that mechanism as the primary mode.

Is it the same host that is desired, or the same runtime?

Agree it's the same runtime. Basically the location we can dotnet exec with to get the same runtime executing the current process.

[agocke]

Is it the same host that is desired, or the same runtime? Given https://github.com/dotnet/designs/pull/303, those could be different locations. My impression is that the same runtime would be the desire.

Actually, I don't think my statement was well-defined. What DOTNET_HOST_PATH is intended to do is point to a muxer that can run the target binary. There's no guarantee that binary will end up actually using the same runtime, due to all the decisions that happen when loading the binary (roll-forward, etc).

I think my proposal is: find the muxer associated with the current hostfxr, if one exists. Run under that muxer. No guarantees about SDK or runtime.

[agocke]

I understand, and agree with, the reluctance for the runtime to be setting environment variables. At the same time though it would seem odd if the runtime is giving an API to find the host executable but then our SDK tool chain is effectively inventing another mechanism via $DOTNET_HOST_PATH environment variable. If we go with a new API then I think we need to consider moving the SDK to using that mechanism as the primary mode.

I think there are potentially two different scenarios here: do you want tools launched under your process to know that this is a .NET process and where it was launched from? Or do you simply want managed processes to be able to find a compatible muxer?

I think of this as the divide between native children and managed children. If you want native children to know the location of the muxer, you probably want to use an environment variable. But also, I don't think this is an appropriate contract to use for the .NET runtime -- we wouldn't want to automatically pollute the environment variable space of child processes.

Conversely, a managed API is effectively an entirely internal contract to a .NET binary. That's definitely the kind of thing that makes sense to build into every .NET app.

So I think an AppContext switch makes a lot of sense as a universal host feature, and environment variable less so. But, I would also find it reasonable if the dotnet CLI/SDK, in particular, wanted to have all child processes inherit a .NET-specific variable. That would be something unique to running tools under the SDK and we wouldn't expect, say, a random WinForms app to start passing its muxer path to native child processes.