Creating a dotnet tool with multiple executables

[ldennington]

Note: This is more of a request for help with a specific scenario than a bug. In light of that, if there's somewhere better for me to direct this request, just let me know.

Describe the bug

My team is attempting to package our Git Credential Manager tool as a dotnet tool (you can find details as to why in this issue). However, while experimenting with this idea, I've been running into issues due to GCM's unique architecture, in which the main exe dynamically discovers and calls a number of helper exes for UI functionality.

To Reproduce

git clone https://github.com/ldennington/git-credential-manager.git
cd git-credential-manager/src
git checkout test-dotnet-tool
dotnet build --configuration MacRelease

GCM currently uses dotnet publish to create and gather the main and UI exes for other types of packaging (for example, this script runs the correct publish commands before this script creates our .pkg file). I've been running a slightly tweaked version of the first script to generate my "payload" for the NuGet package next to the src/shared/GitCredentialManager.csproj file:

.src/osx/Installer.Mac/layout.sh --output=./shared/Git-Credential-Manager/payload

I've modified the Git-Credential-Manager.csproj file to use a .nuspec file that copies everything from payload into the tools/net6.0/any directory:

dotnet pack ./shared/Git-Credential-Manager --output ./shared/Git-Credential-Manager/out/nupkg --no-restore --no-build
dotnet tool install --global --add-source ./shared/Git-Credential-Manager/out/nupkg git-credential-manager-core --prerelease

Running which git-credential-manager-core confirms the tool is in the correct place and on the PATH:

ldennington@Lessleys-MacBook-Pro:~/repos/git-credential-manager/src(test-dotnet-tool⚡) » which git-credential-manager-core    
/Users/ldennington/.dotnet/tools/git-credential-manager-core

9. However, attempting an operation that will trigger the authentication UI (e.g. pushing to a private GitHub repo), results in GCM being unable to find the helper exe and falling back to terminal prompts for username and password auth:

ldennington@Lessleys-MacBook-Pro:~/repos/git-credential-manager/src/shared/Git-Credential-Manager(test-dotnet-tool⚡) » git push -f
Username for 'https://github.com':

Please execute the above command in a terminal outside of VS Code. Additionally, you may have to remove any stored GitHub credentials from keychain for this to repro correctly.

I'm wondering whether you have suggestions or guidance for how to make GCM recognize bundled helper exes after being installed as a dotnet tool.

Exceptions (if any)

N/A

Further technical details

• Include the output of dotnet --info

  
  ldennington@Lessleys-MacBook-Pro:~/repos/git-credential-manager/src/shared/Git-Credential-Manager(test-dotnet-tool⚡) » dotnet --info
  .NET SDK (reflecting any global.json):
  Version:   6.0.201
  Commit:    ef40e6aa06

Runtime Environment: OS Name: Mac OS X OS Version: 12.4 OS Platform: Darwin RID: osx.12-x64 Base Path: /usr/local/share/dotnet/sdk/6.0.201/

Host (useful for support): Version: 6.0.3 Commit: c24d9a9c91

.NET SDKs installed: 3.1.419 [/usr/local/share/dotnet/sdk] 3.1.420 [/usr/local/share/dotnet/sdk] 5.0.405 [/usr/local/share/dotnet/sdk] 5.0.408 [/usr/local/share/dotnet/sdk] 6.0.201 [/usr/local/share/dotnet/sdk]

.NET runtimes installed: Microsoft.AspNetCore.App 3.1.25 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.26 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.10 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.14 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.17 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 3.1.25 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.26 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.10 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.14 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.17 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs: https://aka.ms/dotnet-download

- The IDE (VS / VS Code/ VS4Mac) you're running on, and its version

VS Code, version 1.70.2

[marcpopMSFT]

I understand that you are bundling the additional executables into the nupkg file and those are getting downloaded. When we install a tool, we create an apphost into a different folder. You can probably temporarily fix this by copying those helper binaries into the location of the apphost. Are you perhaps trying to load the helper assemblies from the process executing rather than the location of the tool .dll being launched as you might be able to get those loading by finding them that way.

I'm not sure if the right answer is for us to auto-detect other executables in the same folder as the tool and also create additional apphosts for those or if we should create tool dependency chains where one tool depends on another one and we install both. Both are a bit more involved for us and having you find a way of loading the assemblies with the current architecture would be better.

[ldennington]

Hi @marcpopMSFT! Thanks for your response. I'm totally fine with the strategy of copying the helper binaries into the location of the apphost (~/.dotnet/tools on macOS/Linux, correct?). Do you have suggestions for how to do this though? Is there a way to link a script (maybe in the custom nuspec?) that will copy over the binaries after the tool install has completed?

[KalleOlaviNiemitalo]

IMO, if the additional executables are not declared as commands in DotNetToolSettings.xml, then dotnet tool should not create apphosts for them in any directory that is in the user's PATH by default. But it would be ok to create apphosts in some tool-specific directory and have the main apphost add that directory to PATH of the tool process.

[marcpopMSFT]

Adding @baronfel as I'm not sure of a good way to solve this. Not sure if the tool itself can copy the files around before trying to load them or if there's another way to include a script with the tool nuget package that customers could run (how would they know to run it).

[ldennington]

I believe I actually got a prototype working today using a custom NuGet config. I built it on macOS so will need to confirm tomorrow it works as expected on Linux, but assuming all goes well will plan to close this issue out after I've done that work.

[marcpopMSFT]

Glad to hear it. If you do get it working, please link your solution here so others can build off of what you learned.

[EraYaN]

Would it be possible to update DotnetToolSettings.xml in some way without manually patching the zip file after packing? We create "meta" packages containing many tools (as project references) in one nuget package, and it would be great if they all end up in the DotnetToolSettings.xml automatically or through some option.