Recommendations for new structure of NuGet packages for .NET projects

[Taritsyn]

I have already written about disadvantages of existing packages and gave several recommendations for improving it (see the “Recommendations for improving an existing NuGet packages for .NET projects” issue). In this post, I will write about more radical changes - new structure of NuGet packages. Currently, .NET developers using the ChakraCore library have only one official package available - Microsoft.ChakraCore, which has two significant disadvantages:

1. This package is focused exclusively on Windows and contains assemblies for the following processor architectures: x86, x64 and ARM. Ability to build a library for the ARM64 architecture has existed for at least six months, but nevertheless the corresponding assembly is not included in either official releases or NuGet package.
2. Now this package weighs about 7 MB. If we add assemblies for three most popular platforms (Windows (ARM64), Linux (x64) and OS X (x64)) to it, then it will already weigh more than 40 MB, which may cause problems for some users.

These problems can be solved by creating separate packages for each platform and converting the Microsoft.ChakraCore package to a metapackage. Some developers, including me, have done so by publishing their own packages: JavaScriptEngineSwitcher.ChakraCore.Native.*, BaristaLabs.BaristaCore.ChakraCore.* and ReactWindows.ChakraCore.ARM64. Appearance of official packages, created according to similar principles, would led to three positive points:

1. Removing additional burden from developers of unofficial packages.
2. Increasing trust of end users to packages.
3. Increasing number of downloads of official packages.

I will list the main steps that you need to do to create such packages (examples still use the old package naming system):

1. Move a source code of each NuGet package to separate directory. At the moment, all .nuspec and .props files are located in the same directory. It would be more optimal to create a separate directory for each package and move these files there. In the future, this would make it easier to add new files to packages (for example, PowerShell scripts).
2. Use a RID's in NuGet package names. There are already three platform-specific packages, but they just haven't been published. Since we are also planning to create packages for operating systems other than Windows, then we should use a runtime identifiers in its names. I suggest renaming the platform-specific packages as follows:
   • Microsoft.ChakraCore.X86 -> Microsoft.ChakraCore.win-x86
   • Microsoft.ChakraCore.X64 -> Microsoft.ChakraCore.win-x64
   • Microsoft.ChakraCore.ARM -> Microsoft.ChakraCore.win-arm
3. Implement in MSBuild scripts of platform-specific packages a same deployment mechanism as in the Microsoft.ChakraCore package. Currently, platform-specific packages cannot be used together, because their assemblies overwrite each other. To prevent this, in the .props files you need to use a code similar to code from the Microsoft.ChakraCore.props file.
4. Add a platform-specific “symbol” packages. Having looked at the NuGet.org site many packages that contain native assemblies, I did not find a no one package for which symbols were loaded. It seems that for packages with native assemblies, can't create symbol packages in either old (.symbols.nupkg) or new (.snupkg) format. Therefore, to create a platform-specific “symbol” packages, we will use the same approach as in the Microsoft.ChakraCore.Symbols package: create a regular NuGet packages that will deploy the .pdb files instead of the .dll files.
5. Convert a Microsoft.ChakraCore and Microsoft.ChakraCore.symbols packages to metapackages. Since we now have primary and “symbol” platform-specific packages, then we can convert the Microsoft.ChakraCore and Microsoft.ChakraCore.symbols packages to metapackages. To do this, we will simply remove the logic from these packages and add references to platform-specific packages as dependencies. We keep these packages only for backward compatibility and they will always be intended for Windows.
6. Add to platform-specific packages a PowerShell scripts that are responsible for deploying assemblies and .pdb files on ASP.NET 4.X websites. Unfortunately, MSBuild scripts are not suitable for all use cases of .NET Framework. For example, ASP.NET 4.X web sites don't have project files and need to use a PowerShell scripts to deploy the native assemblies in the bin directory. To make it possible to use packages on ASP.NET 4.X web sites, we will add the Install.ps1 and Uninstall.ps1 scripts to platform-specific packages.
7. Add a support for Windows (ARM64) platform. To create primary and “symbol” packages oriented on the ARM64 processor architecture, you can take the Microsoft.ChakraCore.win-arm.* packages as a basis and just add the 64 numbers in right places. In sample code, I also added the created packages to metapackages as dependencies, as this seemed reasonable to me.
8. Add a Windows batch file that runs the PowerShell script to create NuGet packages (optional). Since running the package.ps1 script by using Windows PowerShell requires certain skills, then I renamed it to the pack.ps1 and created a batch file (pack.cmd) to simplify this procedure.
9. Add a NuGet packages for Linux (x64) and OS X (x64). Packages containing assemblies for 64-bit Linux and OS X are extremely simple, because they are intended only for .NET Core. Because .NET Framework is not supported on Linux and OS X, then to run the nuget.exe need to install the Mono. A simple Bash script (pack.sh) is used to run the packaging process. Ability to create a “symbol” packages is not yet available.

There is also an alternative way to run the packaging process, when using only one script (pack.ps1) written for the cross-platform PowerShell. Running a script by using this version of PowerShell looks much easier:

pwsh pack.ps1

Everything described in this post is already available as a ready-made solution for creating a NuGet packages. Moreover, it is available in two variants: standard and one-script. I am ready to implement each of these recommendations in the form of pull request.

P.S.: I did not touch the Microsoft.ChakraCore.SignNuget.proj and packages.config files, because I do not know exactly how they are used. Most likely, they are used to install the MicroBuild.Core and SrcSrv packages, but it is better to check with Microsoft employees.

[rhuanjl]

I like the idea of supporting platform selection better, also keen on adding macOS and linux binaries to any release channel - want it to be clear that ChakraCore is not just a windows option.

Note we also hope to be able to build ChakraCore on some extra platforms at some point, hopefully within the year but not sure (these capabilities do not yet exist):

• x86 Linux
• ARM Linux
• ARM64 Linux
• apple silicon (arm macOS)

That aside obviously we'll need to drop the Microsoft name from these packages - need to consider an appropriate alternative, possibly chakra-core (as that's our github org name now) though chakra-core.ChakraCore is a bit of an odd package name.

One point that jumps out to me from reading through these examples is the sheer quantity of it - particularly with some duplication for the various components - if/when we need to update aspects of this would it be better to have a powershell or python script that produces all of these nuspec documents rather than updating them manually? (not sure if this is realistic at all but thought it worth mentioning).

@Fly-Style @ppenzin any thoughts on above?

[ppenzin]

@Taritsyn thank you for the detailed instructions. I agree we should do it (it would obviously be a pain, but we only have to do it once).

@rhuanjl regarding the package name, how about ChakraEngine.ChakraCore or Chakra.ChakraCore?

[Taritsyn]

@rhuanjl

I like the idea of supporting platform selection better, also keen on adding macOS and linux binaries to any release channel - want it to be clear that ChakraCore is not just a windows option.

For .NET development, it is already standard practice to create separate packages with the native assemblies for each platform, but what about the C ++ package? Even now, the Microsoft.ChakraCore.vc140 package is quite heavy (it weighs more than 100 MB). and if you add binaries for the ARM64 processor architecture, then it will weigh even more. Most likely it should also be splited into several NuGet packages according to processor architectures, but in this case will have to extract the header files into separate package.

Note we also hope to be able to build ChakraCore on some extra platforms at some point, hopefully within the year but not sure (these capabilities do not yet exist):

At the moment, main problem is that by name of the directory into which the assemblies are copied, it is impossible to understand for which platform build was made:

out/Release/libChakraCore.so
out/Release/libChakraCore.dylib

That aside obviously we'll need to drop the Microsoft name from these packages - need to consider an appropriate alternative, possibly chakra-core (as that's our github org name now) though chakra-core.ChakraCore is a bit of an odd package name.

Usually, name of organization or project in the Pascal case style is used as NuGet package ID prefix. Since the organization is already registered on NuGet.org, then the answer is quite obvious.

A much more difficult problem is choosing a sub-prefix, since the sub-prefix for .NET and C ++ will be different. I would suggest the following prefixes and sub-prefixes:

.NET

• ChakraCore. Sub-prefix can be dispensed with, because NuGet was originally designed for .NET and most packages are considered .NET oriented by default.
• ChakraCore.DotNet. This prefix makes it clear that the package is intended for .NET, but, at the same time, some people may think that this is a C# port of the ChakraCore library.
• ChakraCore.Native. This prefix immediately makes it clear that the package contains the native assemblies, but, at the same time, some people may think that the package is intended for C++.

C++

• ChakraCore.Cpp. It is immediately clear that the package is intended for C++. Such a prefix will be a good complement to the ChakraCore.Native prefix.
• ChakraCore.v140. Meaning of such a prefix will not be clear to everyone.

• ChakraCore.Cpp.v140. A good combination of the previous two variants, which brings more specificity.

  One point that jumps out to me from reading through these examples is the sheer quantity of it - particularly with some duplication for the various components - if/when we need to update aspects of this would it be better to have a powershell or python script that produces all of these nuspec documents rather than updating them manually? (not sure if this is realistic at all but thought it worth mentioning).

Good idea! Basically, I can write a script, that generates a temporary .nuspec, .props and .ps1 files based on data and templates, in cross-platform PowerShell.