Publish LeMP as dotnet CLI tool

[dadhi]

This allows to manage the LeMP command line similar to the nuget package (including discovery, restore, update, etc. goodies). Additionally it may be configured as global or as local to the project.

Here I am creating my own LempDotnetTool by wrapping the existing LeMP distro: https://github.com/dadhi/LempTest/blob/master/LempDotnetTool/LempDotnetTool.csproj

I did this just for testing, having the tool already available would be nice. Creating it is simple - just adding couple of properties to the console app project, see the link above.

Consuming it locally requires a separate manifest file (this kind of lame but works): https://github.com/dadhi/LempTest/blob/master/LibWithEcs/.config/dotnet-tools.json

The tool maybe called via MSBuild task like this: https://github.com/dadhi/LempTest/blob/6cb778f2cce21dd1c1cda06bdb0d26ecff8317f8/LibWithEcs/LibWithEcs.csproj#L13

Couple of links: https://docs.microsoft.com/en-us/dotnet/core/tools/global-tools-how-to-create https://docs.microsoft.com/en-us/dotnet/core/tools/local-tools-how-to-use https://github.com/natemcmaster/dotnet-tools https://www.toolget.net/

[dadhi]

@qwertie Hi, are you planning to support it? What are your thoughts on the topic?

[qwertie]

Hi. I must apologize for not responding, especially the second time (gmail put it in a category I don't look at often, so I have reconfigured it so that future messages are more prominent to me).

Unfortunately, I don't quite understand what you're trying to accomplish with this. There is a NuGet package for LeMP already, but it is no different than any normal package, so it does not, for example, provide anything to help integrate with MSBuild. If you know how to enhance the existing package so that it becomes easier to incorporate LeMP with MSBuild, I would like to know how to do that.

[qwertie]

Hi. I must apologize for not responding, especially the second time (gmail put it in a category I don't look at often, so I have reconfigured it so that future messages are more prominent to me)... okay, scratch that, I apologize for the first time too.

Unfortunately, I don't quite understand what you're trying to accomplish with this. There is a NuGet package for LeMP already, but it is no different than any normal package, so it does not, for example, provide anything to help integrate with MSBuild. If you know how to enhance the existing package so that it becomes easier to incorporate LeMP with MSBuild, I would like to know how to do that.

[dadhi]

@qwertie No problem.

Dotnet CLI tool is the modern go-to way to publish and deploy the console apps in the .NET ecosystem.

There is a separate tab and the search for the tools on the NuGet site, e.g. check the tab .NET CLI here: https://www.nuget.org/packages/Microsoft.DotNet.Watcher.Tools/

In my original mesage I am explaining how I am wrapping and using LeMP existing downloaded distributive as a local dotnet tool. I would propose to publish it directly.

[qwertie]

I'm having difficulty creating a tool package that has a correct version number, description and tool name... It wants to be "version 1.0" basically. Also I'm not sure how to treat it without publishing it first. But I'm... trying.

[dadhi]

You my consume and test the tool locally by adding the folder with packages to nuget sources.

Or you may publish a preview version which I will be happy to test.

Regarding the version you may put anything which makes sense for you. You may synchronize it with the LeMP version or start a new.

You may open the changes with the PR and I can help with the review.

Here is the one of my tools to use as an example https://github.com/dadhi/CsToMd/blob/master/CsToMdDotnetTool/CsToMdDotnetTool.csproj

[qwertie]

You don't understand. I'm saying the version number always comes out as 1.0.0, I can't figure out how to control it.

[dadhi]

Ok, how are you specifying it?

I am specifying the version as following in the .csproj file:

<PropertyGroup>
...
    <VersionPrefix>1.2.3</VersionPrefix>
    <VersionSuffix>preview-01</VersionSuffix>
...

or like this in the .nuspec:

<metadata minClientVersion="3.3.0">
        ...
        <version>1.2.3-preview-01</version>
        ...

[qwertie]

Well, I was sort of following the directions on this page with some tweaks... but the attributes in the (executable) assembly aren't being transferred to the nupkg file. After that I tried to do it with a nuspec file (no csproj), but the <packageTypes><packageType name="DotnetTool" /></packageTypes> element just seems to be ignored, very strange, and I can't find any complete examples of anybody actually doing this with a nuspec file. There are partial nuspec files in blog posts with "..." in them, but no complete nuspec files. I would rather use the nuspec approach since all my other packages are nuspecs, but it just doesn't work, so I guess I'll try the csproj approach again tomorrow. You seem to be talking about a combined csproj + nuspec approach... how does the csproj make reference to a nuspec, or is it the other way around? I would rather not have to specify a version number in the csproj file because I would have to edit the version number separately from the version number used in all the other packages.

[qwertie]

P.S. I can't comprehend how Microsoft thinks it's okay not to write reference documentation for this stuff. Just a basic tutorial with not nearly enough information for real-life scenarios.

[dadhi]

@qwertie

P.S. I can't comprehend how Microsoft thinks it's okay not to write reference documentation for this stuff. Just a basic tutorial with not nearly enough information for real-life scenarios.

Agreed, the tutorials are scarce and cover only the happy-path hello world examples.

At the moment I am publishing my dotnet-tools via .csproj but when built it generates the "proper" (I believe) .nuspec file and the configs in the obj folder. I would've tried to use this output file as the basis.

For instance, for the https://github.com/dadhi/LempTest/tree/master/LempDotnetTool I have the following:

LempDotnetTool.1.0.0-preview-01.nuspec

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2011/10/nuspec.xsd">
  <metadata>
    <id>LempDotnetTool</id>
    <version>1.0.0-preview-01</version>
    <authors>Maksim Volkau</authors>
    <owners>Maksim Volkau</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>Package Description</description>
    <copyright>Copyright © 2020 Maksim Volkau</copyright>
    <packageTypes>
      <packageType name="DotnetTool" />
    </packageTypes>
  </metadata>
  <files>
    <file src="C:\Code\LempTemplatingTest\LempDotnetTool\obj\DotnetToolSettings.xml" target="tools\netcoreapp3.1\any\DotnetToolSettings.xml" />
    <file src="C:\Code\LempTemplatingTest\LempDotnetTool\bin\Release\netcoreapp3.1\publish\LempDotnetTool.dll" target="tools\netcoreapp3.1\any\LempDotnetTool.dll" />
    <file src="C:\Code\LempTemplatingTest\LempDotnetTool\bin\Release\netcoreapp3.1\publish\LempDotnetTool.deps.json" target="tools\netcoreapp3.1\any\LempDotnetTool.deps.json" />
    <file src="C:\Code\LempTemplatingTest\LempDotnetTool\bin\Release\netcoreapp3.1\publish\LempDotnetTool.runtimeconfig.json" target="tools\netcoreapp3.1\any\LempDotnetTool.runtimeconfig.json" />
    <file src="C:\Code\LempTemplatingTest\LempDotnetTool\bin\Release\netcoreapp3.1\publish\LempDotnetTool.pdb" target="tools\netcoreapp3.1\any\LempDotnetTool.pdb" />
  </files>
</package>

DotnetToolSettings.xml

<?xml version="1.0" encoding="utf-8"?>
<DotNetCliTool Version="1">
  <Commands>
    <Command Name="lemp" EntryPoint="LempDotnetTool.dll" Runner="dotnet" />
  </Commands>
</DotNetCliTool>

There are also other props, targets and jsons, but it is probably internal stuff.

[qwertie]

Thanks for the tip, but err... my understanding is that these tools must be packaged with all dependencies included. So basically the nupkg should be over 7 MB, so the list of files in your nuspec is way too short. But I've located the nuspec produced by Visual Studio in obj\Debug and it has 86 <file> elements. I'll try modifying this nuspec for my needs - in particular I want to avoid spelling out every Roslyn resource assembly and dependency by hand... I've seen it claimed that <dependency...> elements don't work for these tools - supposedly because they want tools to be self-contained - nuget pack could automatically reformulate the <dependency...> elements in the nupkg as if they were <file...> elements, but maybe they didn't think of that or judged it too difficult.

[qwertie]

So now I have to worry about the fact that the Roslyn assemblies are not being copied to my output folder during build, so I can't copy them from the output folder. When I debug LeMP, Roslyn DLLs are somehow being loaded from folders like C:\Users\Family.nuget\packages\microsoft.codeanalysis.csharp.scripting\3.6.0\lib\netstandard2.0 which is baffling, I mean, what is causing .NET Core 2.1 to look inside .nuget to find DLLs? The problem for writing a nuspec is that I obviously can't reference C:\Users\Family in the nuspec file, as it would only build correctly on my own machine. There is another copy of the nuget packages inside a "packages" folder in the repo, but even if I can count on other copies of the repo getting packages restored here (and I have no idea if that's a safe assumption), I don't want to refer to folders like "Microsoft.CodeAnalysis.Common.3.6.0" because if I upgrade Roslyn later, I will have to remember to upgrade all the references in the nuspec or it will silently break. (In fact I already had this sort of problem without noticing. The original LeMP package is partially broken because I changed it from .NET Standard 2.0 to .NET Core 2.1, and since the nuspec copies from a Release\netstandard2.0 folder instead of netcoreapp2.1, it only picks up the macros DLL which is still targeting .NET Standard.)

[qwertie]

At least I figured out why .NET is able to find packages in C:\Users\Family\.dotnet. Visual Studio (or something) generates a magic file called LeMP.runtimeconfig.dev.json which points to it:

{
  "runtimeOptions": {
    "additionalProbingPaths": [
      "C:\\Users\\Family\\.dotnet\\store\\|arch|\\|tfm|",
      "C:\\Users\\Family\\.nuget\\packages"
    ]
  }
}

I decided to use a feature of AppVeyor (namely I can use powershell commands inside its config file) to grab Microsoft.CodeAnalysis*.dll from nuget packages so they could be used in the new package. But now I have a new problem: without the magic paths in LeMP.runtimeconfig.dev.json, there are more ordinary .NET Core dlls that don't load:

PS C:\Dev\ecsharp\LeMP-tool.28.2.0-ci291.nupkg> dotnet .\tools\netcoreapp2.1\any\LeMP.dll temp.ecs
Error:
  An assembly specified in the application dependencies manifest (LeMP.deps.json) was not found:
    package: 'System.Runtime.CompilerServices.Unsafe', version: '4.7.0'
    path: 'lib/netcoreapp2.0/System.Runtime.CompilerServices.Unsafe.dll'

Apparently, a lot of "System" DLLs are, like, add-ons rather than built into .NET Core, and Roslyn uses some of these as nuget packages. So probably I can get this working after I repeat a similar process for these DLLs as I did for Roslyn.

Something is strange about Microsoft.CodeAnalysis.Analyzers, which is one of the dependencies of Microsoft.CodeAnalysis.Common according to LeMP.deps.json: instead of a "lib\netstandard2.0" folder, it contains a "analyzers\dotnet\cs" folder. I'm going to ignore that one and hope for the best.

[qwertie]

DLL hell ... solved.

Error:
  An assembly specified in the application dependencies manifest (LeMP.deps.json) was not found:
    package: 'System.Text.Encoding.CodePages', version: '4.5.1'
    path: 'runtimes/win/lib/netcoreapp2.0/System.Text.Encoding.CodePages.dll'

One of the dependencies has a bizarre requirement to be located in a sequence of subfolders: runtimes/win/lib/netcoreapp2.0

I have never seen anything like this in .NET-land before. It's some kind of new .NET Core thing. But it led me to google around and find a relatively simple solution to this whole mess: dotnet publish will give me the files I need prior to nuget pack.

dotnet publish -c Release "Main\LeMP\LeMP.csproj" --no-build -o .\Bin\published

So I have finally published the nuget package as "LeMP-tool". Give it a try!

[dadhi]

Great, thanks, will try.

[dadhi]

Works great! Closing the issue.