Make a global .NET CLI tool from GCRealTimeMon

chrisnas commented 2 years ago

In order to make it easy to deploy GCRealTimeMon on machines, make it as a .NET CLI global tool. To be consistent with the existing CLI tools (dotnet gcheap/dump/trace/counters), the chosen name is dotnet-gcmon but could easily be changed.

The folder hierarchy has been changed to keep both console applications while sharing the code

Another option would be to:

Wrap the current code in Main() into a sharable Run() method of a new public class
Use it both in GCRealTimeMon and dotnet-gcmon as a reference This solution avoids the need to update dotnet-gcmon each time a new file is added into GCRealTimeMon (BTW this is what I have done for pstacks/dotnet-pstacks - https://github.com/chrisnas/DebuggingExtensions/tree/master/src/dotnet-pstacks)

Last but not least, the current version is 0.1.0 (see readme.txt for more details about publishing the tool)

Maoni0 commented 2 years ago

thank you @chrisnas! :) this is great! we should probably just call it gcmon, not dotnet-gcmon so when you do the dotnet command you do dotnet gcmon -p pid instead of dotnet dotnet-gcmon -p pid

which seems cleaner, what do you think? (or we could call it gcrealmon just to emphasize the real time property of it)

I cloned your repo and did dotnet publish -c Release -r win-x64 which gave me this error -

  GCRealTimeMon -> C:\realmon\src\GCRealTimeMon\bin\Release\net6.0\GCRealTimeMon.dll
  dotnet-gcmon -> C:\realmon\src\dotnet-gcmon\bin\Release\net6.0\win-x64\dotnet-gcmon.dll
  GCRealTimeMon -> C:\realmon\src\GCRealTimeMon\bin\Release\net6.0\win-x64\GCRealTimeMon.dll
  GCRealTimeMon.UnitTests -> C:\realmon\tests\GCRealTimeMon.UnitTests\bin\Release\net6.0\win-x64\GCRealTimeMon.UnitTests.dll
  GCRealTimeMon -> C:\realmon\src\GCRealTimeMon\bin\Release\net6.0\win-x64\publish\
C:\Program Files\dotnet\sdk\6.0.100-rc.2.21505.57\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.PackTool.targets(86,5): error NETSDK1053: Pack as tool does not support self contained. [C:\realmon\src\dotnet-gcmon\dotnet-gcmon.csproj]
  GCRealTimeMon.UnitTests -> C:\realmon\tests\GCRealTimeMon.UnitTests\bin\Release\net6.0\win-x64\publish\

any ideas?

also I've been wondering whether we should even keep a separate GCRealTimeMon dir anymore. this is what I think would be good ways for folks to use this project -

for people who want to just use the tool, it's great if they can get it from nuget like all the other dotnet CLI tools (I am a little disappointed to find out that nuget does not support the AoTed version... the mere fact of running dotnet install would mean you'd need the SDK installed whereas the AoTed version wouldn't need anything... but I've been told that publishing binaries directly isn't recommended....)
for people who want to participate in the development of the tool, if they already have VS, it'd be nice to be able to do their development in VS. they could then generate the AoTed version for their coworkers or to run on other machines.

so at the end of the day this means you'd need to have the .net SDK installed. can you think of another scenario how people use this tool?

chrisnas commented 2 years ago

thank you @chrisnas! :) this is great!

You're welcome @Maoni0 :^)

we should probably just call it gcmon, not dotnet-gcmon so when you do the dotnet command you do dotnet gcmon -p pid instead of dotnet dotnet-gcmon -p pid which seems cleaner, what do you think?

This is expected: the nuget has to be called "dotnet-gcmon" so that it can be used as "dotnet gcmon" :^)

(or we could call it gcrealmon just to emphasize the real time property of it)

It is up to you: I tried to stick to the "dotnet counters" or "dotnet trace" naming without "real"

I cloned your repo and did dotnet publish -c Release -r win-x64 which gave me this error -

  GCRealTimeMon -> C:\realmon\src\GCRealTimeMon\bin\Release\net6.0\GCRealTimeMon.dll
  dotnet-gcmon -> C:\realmon\src\dotnet-gcmon\bin\Release\net6.0\win-x64\dotnet-gcmon.dll
  GCRealTimeMon -> C:\realmon\src\GCRealTimeMon\bin\Release\net6.0\win-x64\GCRealTimeMon.dll
  GCRealTimeMon.UnitTests -> C:\realmon\tests\GCRealTimeMon.UnitTests\bin\Release\net6.0\win-x64\GCRealTimeMon.UnitTests.dll
  GCRealTimeMon -> C:\realmon\src\GCRealTimeMon\bin\Release\net6.0\win-x64\publish\
C:\Program Files\dotnet\sdk\6.0.100-rc.2.21505.57\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.PackTool.targets(86,5): error NETSDK1053: Pack as tool does not support self contained. [C:\realmon\src\dotnet-gcmon\dotnet-gcmon.csproj]
  GCRealTimeMon.UnitTests -> C:\realmon\tests\GCRealTimeMon.UnitTests\bin\Release\net6.0\win-x64\publish\

any ideas?

I guess this is because you need to be AnyCPU to support all platforms I don't get the error because I'm not using -r win-x64 just -c Release (like I explain in the readme.txt for dotnet pack)

also I've been wondering whether we should even keep a separate GCRealTimeMon dir anymore. this is what I think would be good ways for folks to use this project -

for people who want to just use the tool, it's great if they can get it from nuget like all the other dotnet CLI tools (I am a little disappointed to find out that nuget does not support the AoTed version... the mere fact of running dotnet install would mean you'd need the SDK installed whereas the AoTed version wouldn't need anything... but I've been told that publishing binaries directly isn't recommended....)

for people who want to participate in the development of the tool, if they already have VS, it'd be nice to be able to do their development in VS. they could then generate the AoTed version for their coworkers or to run on other machines.

so at the end of the day this means you'd need to have the .net SDK installed.

The fact that you need to first install the SDK to be able to "automatically" install a tool was a large subject of discussion with the Diagnostics team. And it led to one of my PRs: the creation of the downlad pages in MSDocs for direct access without SDK like https://docs.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-dump ;^)

You're right about the AoTed version: it would be much easier to have a single .exe to copy... However, I think that you also need a .yaml file or the app crashes. A simple zip to copy to a folder that is in the PATH would be enough to use the tool... but with "dotnet-gcmon" instead of "dotnet gcmon". Not that hard to use a "-" instead of a SPACE.

can you think of another scenario how people use this tool?

I would use this tool in different ways:

define my own set of metrics that I know help me to pinpoint issues in my code such as suspension time, compacting gen2, increase in gen2 or LOH for leaks.
generate kind of "gc logs" that could be analyzed post mortem like what you do in perfview but more easily (even write some code to process the output if saved as .csv file like what "dotnet counters collect" provides)

It would be a formidable tool to use in your https://github.com/Maoni0/mem-doc: explaining how different issues become visible through which metrics!

Maoni0 commented 2 years ago

This is expected: the nuget has to be called "dotnet-gcmon" so that it can be used as "dotnet gcmon" :^)

your screen shot on twitter says dotnet dotnet-gcmon.dll thus my question - can this be dotnet gcmon.dll instead (or dotnet gcrealmon.dll). but I think you are saying it's already dotnet gcmon.dll?

I don't get the error because I'm not using -r win-x64 just -c Release (like I explain in the readme.txt for dotnet pack)

interesting - I didn't even notice the realme.txt (I just cloned your repo and tried to build...haven't actually gone to see the details yet). do you see a problem folding the content of this readme.txt into the exiting readme.md?

is the idea why we have a separate dotnet-gcmon dir to build the global CLI tool so we don't affect the "normal" build?

Maoni0 commented 2 years ago

I just took a look at the readme.txt, I'm confused about the build instructions there. it says

It is also possible to manually generate the package in Release, by using the following command in the .csproj folder:
   dotnet pack -c Release

what's the "normal" way to build this? by loading the .csproj in VS? by some other dotnet commandline?

chrisnas commented 2 years ago

This is expected: the nuget has to be called "dotnet-gcmon" so that it can be used as "dotnet gcmon" :^)

your screen shot on twitter says dotnet dotnet-gcmon.dll thus my question - can this be dotnet gcmon.dll instead (or dotnet gcrealmon.dll). but I think you are saying it's already dotnet gcmon.dll?

In my tweet I used the generated dotnet-gcmon.dll to test that it runs on Windows and Linux. Since I did not publish it, I could not "show" what would be the end result (i.e. dotnet gcmon)

I don't get the error because I'm not using -r win-x64 just -c Release (like I explain in the readme.txt for dotnet pack)

interesting - I didn't even notice the realme.txt (I just cloned your repo and tried to build...haven't actually gone to see the details yet). do you see a problem folding the content of this readme.txt into the exiting readme.md?

No problem to update the existing readme.md

is the idea why we have a separate dotnet-gcmon dir to build the global CLI tool so we don't affect the "normal" build?

Yes: that way, the existing GCRealTimeMon console app stays at it is (except if you want to only keep dotnet-gcmon). And now, you can also generate the nuget package that once uploaded on nuget.org will allow everyone to:

install it with: dotnet install -g dotnet-gcmon
run it with: dotnet gcmon

chrisnas commented 2 years ago

I just took a look at the readme.txt, I'm confused about the build instructions there. it says
It is also possible to manually generate the package in Release, by using the following command in the .csproj folder:
   dotnet pack -c Release
what's the "normal" way to build this? by loading the .csproj in VS? by some other dotnet commandline?

Yes: the .csproj contains the property that automatically generates the nuget package each time it is built (either in VS or via dotnet/msbuild command line)

Maoni0 commented 2 years ago

I see. I just tried and built the dotnet-gcmon proj and I do see a .nkpkg file get created -

1>Successfully created package 'C:\realmon\src\dotnet-gcmon\nupkg\dotnet-gcmon.0.1.0.nupkg'.

this is pretty small, ~3mb. so I presume if I upload this via https://www.nuget.org/packages/manage/upload it would also upload the corresponding files.

when I tried to upload this to nuget I got this:

We found the following issue(s):
All published packages should have license information specified. Learn more.
We recommend that you fix these issues and upload a new package. To learn more about authoring great packages, view our Best Practices page.

looks like we need to add a license somewhere... I think anything that allows this to be freely used by anyone who wants to use it is fine with me. if you could add this and edit the readme.md it'd be great!

chrisnas commented 2 years ago

I see. I just tried and built the dotnet-gcmon proj and I do see a .nkpkg file get created -

1>Successfully created package 'C:\realmon\src\dotnet-gcmon\nupkg\dotnet-gcmon.0.1.0.nupkg'.

this is pretty small, ~3mb. so I presume if I upload this via https://www.nuget.org/packages/manage/upload it would also upload the corresponding files.

when I tried to upload this to nuget I got this:
We found the following issue(s):
All published packages should have license information specified. Learn more.
We recommend that you fix these issues and upload a new package. To learn more about authoring great packages, view our Best Practices page.
looks like we need to add a license somewhere... I think anything that allows this to be freely used by anyone who wants to use it is fine with me. if you could add this and edit the readme.md it'd be great!

Would it be fine to use the same MIT licence as other CLI global tools (https://licenses.nuget.org/MIT)? This is what I'm using for dotnet-pstacks (https://github.com/chrisnas/DebuggingExtensions/blob/master/LICENSE) I'll check if anything else is missing before publishing (without publishing)

Maoni0 commented 2 years ago

Would it be fine to use the same MIT licence as other CLI global tools (https://licenses.nuget.org/MIT)?

I would think so :)

chrisnas commented 2 years ago

Would it be fine to use the same MIT licence as other CLI global tools (https://licenses.nuget.org/MIT)?

I would think so :)

I updated the readme.md and added an MIT license file: ready to roll!

chrisnas commented 2 years ago

It seems that I need a "running workflows approval" since it is my first PR on this repo before being able to merge this PR What is the next step?

Maoni0 commented 2 years ago

I've uploaded the .nupkg to nuget. first time doing this. it's asking me to wait...let's see what happens next :D

thanks so much for making this happen, @chrisnas!!🙂

Maoni0 commented 2 years ago

I'm getting an exception because it can't find the config in the local dir... looks like we should change this to read from where gcmon dll is.

C:\temp>dotnet gcmon -p 40360
------- press s for current stats or any other key to exit -------
Unhandled exception. System.IO.FileNotFoundException: Could not find file 'C:\temp\DefaultConfig.yaml'.
File name: 'C:\temp\DefaultConfig.yaml'
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategy(FileStream fileStream, String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize)
   at System.IO.File.AsyncStreamReader(String path, Encoding encoding)
   at System.IO.File.InternalReadAllTextAsync(String path, Encoding encoding, CancellationToken cancellationToken)
   at realmon.Configuration.ConfigurationReader.ReadConfigurationAsync(String path) in C:\realmon\src\GCRealTimeMon\Configuration\ConfigurationReader.cs:line 30
   at realmon.Program.<>c.<<Main>b__11_0>d.MoveNext() in C:\realmon\src\GCRealTimeMon\Program.cs:line 201
--- End of stack trace from previous location ---
   at realmon.Program.Main(String[] args) in C:\realmon\src\GCRealTimeMon\Program.cs:line 193
   at realmon.Program.<Main>(String[] args)

the weird thing is I actually don't see the package in my user's .nuget\packages at all. when I tried to put it under the debugger it always just terminates the process before it loads gcmon dll! but nuget claims it's installed -

C:\temp>\tools\nuget.exe list gcmon
dotnet-gcmon 0.1.0

seems bizarre to me. any ideas? I'm in meetings all day today so I'll check back later.

Maoni0 commented 2 years ago

wait, interesting this was installed at my user's .dotnet\tools\.store\dotnet-gcmon\ dir. I didn't realize this dotnet cmd spawns a new process (actually a few new processes).... I didn't know that nuget would install anything outside the nuget packages dir..

Maoni0 / realmon

Make a global .NET CLI tool from GCRealTimeMon #12