Consider having Chocolatey.lib target .NET standard instead of .NET framework

[xforever1313]

It would be great if chocolatey.lib would be able to target .NET Standard, or multi target .NET Framework and .NET Core. This way, if one needs to embed chocolatey.lib into their application, they are not tied to .NET Framework, but have a choice between any .NET runtime that implements .NET Standard.

[vexx32]

This is planned but we don't have a firm timeline when exactly. Since choco runs many commands in a PowerShell host, I'm not entirely sure we can ever completely target .NET Standard (it might be possible, I'm just not clear on exactly how that would work) but we do plan to update things to have choco be functional on .NET Core.

Related: https://github.com/chocolatey/choco/issues/1990

[tapika]

Related: #2147

[tapika]

I have .net core prototype, let me know if you need one, source code is not public yet at least. (See ticket above)

[TheCakeIsNaOH]

Following along from https://github.com/chocolatey/choco/issues/2436, I have been playing around with building Chocolatey CLI on dotnet7

As noted by @vexx32, Chocolatey uses PowerShell assemblies to setup a PowerShell host. These assemblies are available for either Windows PowerShell (v2,v3,v4,v5,v5.1) on .Net framework, or available for Powershell v6.x/v7.x on Dotnet. The newer PowerShell assemblies appear to be more strongly linked to the dotnet version, e.g. Microsoft.PowerShell.SDK v7.3.x is build for dotnet 7 only.

So therefore, I think it would be very unlikely that chocolatey.lib would be able to be built with .Net standard. With dotnet 6/7/8, sure. With enough refactoring, I think it would be possible to have a chocolatey.slim.lib or whatever that just supports functions which do not need the PowerShell host and is build on .Net standard, but the PowerShell host is currently integrated deep in some of the install logic.

I do have a branch that is currently (kind of) building on dotnet 7: https://github.com/TheCakeIsNaOH/choco/tree/dotnet7 I'm currently struggling with the cake script wanting to run the .net framework restore/build, but it is building (and passing all nunit tests) in Visual Studio.

Some of the changes required don't break too much, and are pretty simple to implement like using the dotnet built in operating system check as a guard (so the VS analyzers understand it). However, there are a number of changes that are more complex:

1. BinaryFormatter is going away in Dotnet. The warning currently can be disabled, but a replacement implementation of the object deep copy will be needed.
2. Threads can no longer be aborted, so the readline/readkey implementations probably will need to be fixed.
3. The alphafs library appears to be mostly abandoned, and has no build for .Net standard/dotnet. I personally made a build for dotnet, which seems to work, but given that NuGet Client now directly uses the build in filesystem calls, the alphafs package can probably just be removed with Chocolatey CLI also just directly using the native calls.
4. The previous method used to get network trace logging no longer works, but there is a new method, which also works on Linux, and would fix https://github.com/chocolatey/choco/issues/2423
5. The current NUnit tests use reflection to change some readonly items in ApplicationParameters, which no longer works in dotnet (for good reason imho), so those have to not have readonly but have a backing field that can be changed. Or another better method for switching things during tests should be devised.
6. As I mentioned above, switching to pwsh is required, specifically pwsh 7.3.x for dotnet 7. This will be a breaking change for some packages/extensions, I just don't know how many.
7. ILMerge has been discontinued, and from what I understand, does not work with dotnet programs/libraries. However, self-contained/ready to run/single file/aot programs are available on dotnet 7, so hopefully a good alternative path can be charted there.
8. Currently, the Mono implementation of ProtectedData is in use for non-Windows platforms. ALso, the dotnet ProtectedData class is only available on Windows, and is not expected to ever be available on other platforms, as it relies on the Windows DPAPI. This likely requires changes in the major version before to start migrating people to whatever the new solution for config obfuscation on non-Windows platforms is. https://github.com/chocolatey/choco/issues/3266

[tapika]

I have also prototyped, and successfully upgraded to .net 6. Fork is available in here:

https://github.com/tapika/swupd

As for 3, I have switched from alphafs to normal file system operations. Not sure if long path problem is still there (path lengths over 260 characters), haven't tested. To my best understanding if path length limitation still exists - that needs to be raised as as bug to .net 6 or later framework.

5th item was simply fixed by removing reflection usage. I know it's also possible to use InternalsVisibleTo attribute:

https://stackoverflow.com/a/9122799/2338477

But at least I'm not in favor of that solution, as it limits who sees what in context of unit testing.

6 is indeed a breakage to some of already published packages. By altering some of powershell helper functions I was able to make it working on some extent in backwards compatible manner. At least readytorun choco.exe can install .net core 3.1 with it's dependencies which works out of box. Haven't tested other packages, but I know some of them does not work.

What I've briefly analyzed - powershell is old, odd and grotesque way of creating package scripts. Much simpler solution would be just to use C# and compile C# scripts manually. Besides basic use cases (like powershell OS queries), it can allow also accessing to UI, see for example this link:

https://www.cs-script.net/cs-script/help-legacy/wpf.html

Simplest approach to run C# script is just to generate C# project (In similar manner to shimgen https://github.com/tapika/swupd/tree/master/src/shimgen), compile and execute it + using some regex to detect special tags as C# script does (css_inc, css_ref and so on).

I have considered already enabling C# scripting in my own choco fork, but haven't started integration of that one yet. C# scripting is not available on my own public github repos, except maybe syncproj, which is outdated. (shimgen has better project generation logic over syncproj, which can have UI access)

7 is re-implemented as ready-to-run self-containing choco.exe executable.

As for 8 - I've removed usage of ProtectedData, See also https://github.com/tapika/swupd/blob/9ad4bb7a819aa7219f95d3420a1e49f096451a8c/src/chocolatey/infrastructure/cryptography/SimpleAES.cs#L13

[tapika]

1 - appears as a warning, maybe something to consider after that interface is removed.

2, I suspect this will do the trick: https://github.com/tapika/swupd/commit/d5f1536cf3c18443c92fa5b435d605fb4df71870

as for 4 - I'm not using mono anymore, replaced with readytorun. Apparently that still exists in my own choco fork, but probably does not work, as mentioned by this item.

[TheCakeIsNaOH]

I've played with a Dotnet 8 build a bit: https://github.com/TheCakeIsNaOH/choco/commits/dotnet-8/ The cake recipe is now much more amenable to running the dotnet 8 build, and it will run and build packages, with all of the NUnit tests passing.

Some minor notes:

• BinaryFormatter is still being removed, but can be worked around with a project flag. It looks like it is mostly just used for the configuation, and a manually written clone should be able to be added to that. So not a dealbreaker, just something to work around later.
• I think I figured out cancelation tokens for the readline/readkey classes, so Threads abort are no longer problematic.
• Alphafs and rhino licensing can both be built pretty much as is in netstandard.
• Just dumping all of the required libraries beside choco.exe seems to work without issue. I'm going to leave figuring out an ilmerge replacement to whoever has to get the licensed extension working.
• Protected data is still an issue, see linked issue above
• Simpleinjector had to be upgrade to v3, with breaking changes. Seems to work with minimal modification, but could be broken.
• PowerShell is another big remaining item.

For PowerShell, there is a very helpful page at: https://learn.microsoft.com/en-us/powershell/scripting/dev-cross-plat/choosing-the-right-nuget-package

Given what I understand from it, when building with Dotnet, then Microsoft.Powershell.SDK is the correct package to use, which locks the hosted script runner into the version(s) of PowerShell available for that version of Dotnet (due to Microsoft.Powershell.SDK). This means running everything on v7.4.x for dotnet 8.

This has a couple of implications:

• The move to PowerShell core removed a bunch of cmdlets. This means a bunch of packages are not compatible with PowerShell v6+. Some of these can be fixed with moving to cmdlets that are equivalent. But that does not work for existing packages. It looks like at least some of these commands can be pollyfilled to the point where they should make a bunch of packages compatible. However, there may be further less obvious incompatibilities that show up and are harder to fix. https://github.com/chocolatey-community/chocolatey-packages/issues/2337
• The new Chocolatey.PowerShell.dll is currently built against the Windows PowerShell assemblies. It can be switched to Microsoft.Powershell.SDK, but that means it is only compatible with that version of PowerShell, which is problematic for both the initial installation of Chocolatey, and for anyone using the external host. There are a couple of workarounds for this. One would be to have multiple build targets, and load the appropriate version in the .psm1, which would work but adds a bunch of complexity during build and testing. Another would be to remove the external host feature, and rework the initial installation so the chocolateyinstall.ps1 of Chocolatey CLI is not run by the install script directly, but instead hosted inside choco.exe. The third solution, and the one I settled on was to build against the PowerShell Standard v3 assemblies with .Net 4.6.1. This allows the build assembly to be used any version of Powershell, v3 on up to v7 (and hopefully beyond), so the current install script still works (without changes), and both the internal and external hosts are happy (as far as I have tested so far). The downside of this is that it brings the requirement to a firm v3, and that only prerelease assemblies are available for v3 (v5.1 release assemblies are available, which would be an option once the minimum version is ok to be v5.1).
• There appear to be a few issues in the functions in the chocolateyInstall.psm1, still working on that.
• There are a number of calls to powershell.exe, like in the external host, and in Start-ChocolateyProcessAsAdmin, still looking at these.

[tapika]

Good to see some progress after maybe 2 years changes made by me in my own fork.

I'm going to leave figuring out an ilmerge replacement to whoever has to get the licensed extension working.

I've used ready2run instead of ilmerge. ilmerged executable still requires .net core installation, while ready2run can work without any preinstallation.

I've abandoned by fork so far, as did not see any need on your side, as well as on my own side. Don't have any intention to revive it, at least for now.

[tapika]

Alphafs and rhino licensing can both be built pretty much as is in netstandard.

I've understood that alphafs was used because of network path limitation removal. Maybe better approach would be to raise this as a bug to .net core itself (raise as a issue to https://github.com/dotnet/runtime) and wait until it gets fixed.