nccs should support /reference command line option like csc.exe

[devhawk]

Currently, nccs uses a hard coded list of references in CompilationContext.Compile when compiling a smart contract. While having a list of hard coded references to use as a default is reasonable, it should also be possible to pass the list of metadata references via the command line using the /reference or -reference command line option as csc does. MSBuild already has the capacity to calculate the set of required references and passing that set to the compiler via the command line.

At this time, it's unclear how nccs should handle project references beyond the ones included in the current hardcoded list. #546 implies that nccs will be using the source code of Neo SmartContract Runtime package. Is a similar approach going to be supported for other references that build on Neo SmartContract Runtime? For example, could I ship a package as a 3rd party named DevHawk.Neo.Contract.Helpers that other developers could include in their smart contract projects? Would the DevHawk.Neo.Contract.Helpers package need to include source code in the package as #546 is doing? Will nccs be able to include the source from the DevHawk.Neo.Contract.Helpers package in the compilation of the contract?

[erikzhang]

For example, could I ship a package as a 3rd party named DevHawk.Neo.Contract.Helpers that other developers could include in their smart contract projects?

Yes, you can.

Would the DevHawk.Neo.Contract.Helpers package need to include source code in the package as #546 is doing?

If the methods in your library contains a method body, then you must include the source code. If all methods are extern methods, then there is no need to include source code.

[devhawk]

@erikzhang I'll be happy to handle fixing #551 if you handle fixing #550

[erikzhang]

@devhawk Can you provide an example how to use /reference?

[devhawk]

@devhawk Can you provide an example how to use /reference?

MSBuild calculates the full list of references that the C# compiler will need to compile a given project. It handles the logic needed to locate core .net references like System.Runtime and System.Numerics, package references like Neo.SmartContract.Framework and project references. The paths to these references are passed to csc via the /reference command line option.

Below, I've included a subset of the full command line MSBuild passes to csc when compiling the registrar sample. (full command line attached as a file)

nccs must keep the logic to resolve the common references and SmartContract.Framework for the CompileSources scenario. What I'm recommending is that nccs generates the list of MetadataReferences from the command line if one or more /reference options are provided. This would enable us to build NccsTask (aka #551) and leverage existing MSBuild logic for calculating the complete set of references, packages and projects.

C:\Program Files\dotnet\dotnet.exe exec "C:\Program Files\dotnet\sdk\5.0.202\Roslyn\bincore\csc.dll" 
/noconfig /unsafe- /checked- /nowarn:1701,1702,1701,1702 /fullpaths /nostdlib+ /errorreport:prompt /warn:5 
/define:TRACE;DEBUG;NET;NET5_0;NETCOREAPP;NET5_0_OR_GREATER;NETCOREAPP1_0_OR_GREATER;NETCOREAPP1_1_OR_GREATER;NETCOREAPP2_0_OR_GREATER;NETCOREAPP2_1_OR_GREATER;NETCOREAPP2_2_OR_GREATER;NETCOREAPP3_0_OR_GREATER;NETCOREAPP3_1_OR_GREATER /highentropyva+ 
/reference:"C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\5.0.0\ref\net5.0\Microsoft.CSharp.dll" 
/reference:"C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\5.0.0\ref\net5.0\Microsoft.VisualBasic.Core.dll" 
/reference:"C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\5.0.0\ref\net5.0\Microsoft.VisualBasic.dll" 
/reference:"C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\5.0.0\ref\net5.0\Microsoft.Win32.Primitives.dll" 
/reference:"C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\5.0.0\ref\net5.0\mscorlib.dll" 
/reference:C:\Users\harry\.nuget\packages\neo.smartcontract.framework\3.0.0-rc1\lib\net5.0\Neo.SmartContract.Framework.dll 
<about 150 more /reference options>
/debug+ /debug:portable /filealign:512 /optimize- /target:library /warnaserror- /utf8output /deterministic+ /langversion:9.0 /warnaserror+:NU1605
/out:obj\Debug\net5.0\registrar.dll /refout:obj\Debug\net5.0\ref\registrar.dll /analyzerconfig:obj\Debug\net5.0\registrar.GeneratedMSBuildEditorConfig.editorconfig /analyzerconfig:"C:\Program Files\dotnet\sdk\5.0.202\Sdks\Microsoft.NET.Sdk\analyzers\build\config\AnalysisLevel_5_Default.editorconfig" 
/analyzer:"C:\Program Files\dotnet\sdk\5.0.202\Sdks\Microsoft.NET.Sdk\targets\..\analyzers\Microsoft.CodeAnalysis.CSharp.NetAnalyzers.dll" /analyzer:"C:\Program Files\dotnet\sdk\5.0.202\Sdks\Microsoft.NET.Sdk\targets\..\analyzers\Microsoft.CodeAnalysis.NetAnalyzers.dll" 
Registrar.cs 
"obj\Debug\net5.0\.NETCoreApp,Version=v5.0.AssemblyAttributes.cs" 
obj\Debug\net5.0\registrar.AssemblyInfo.cs 

[erikzhang]

nccs must keep the logic to resolve the common references and SmartContract.Framework for the CompileSources scenario.

If so, we have kept most of the logic of analyzing .csproj and project.assets.json. I doubt whether it is meaningful to accept the /reference parameters.

[devhawk]

If so, we have kept most of the logic of analyzing .csproj and project.assets.json. I doubt whether it is meaningful to accept the /reference parameters.

As I understand it, CompileSources is used for compiling csharp source files directly - i.e. nccs MyContract.cs while CompileProject is used for compiling .csproj files (though most of the .csproj and project.assets.json parsing logic is in GetCompilation).

GIven the size of the MSBuild code base, I'm confident that we don't want to try and replicate that logic in CompileProject and GetCompilation.

I am suggesting that we keep the commonReferences logic used in CompileSources for scenarios where the developer passes no /references on the command line but replace the logic in CompileProject and GetCompilation with the simpler list of /references provided via the command line. However, I would also suggest that we add the /references support first and then remove the CompileProject/GetCompilation logic in a later PR.

[erikzhang]

The problem is that we need to use the source code of the Neo.SmartContract.Framework package or any other packages. That means we have to analyze project.assets.json. If we only support /references, we won't be able to read the source codes of the references.

[devhawk]

The problem is that we need to use the source code of the Neo.SmartContract.Framework package or any other packages. That means we have to analyze project.assets.json. If we only support /references, we won't be able to read the source codes of the references.

MSBuild provides the ResolveAssemblyReferences task that computes the full closure of references used in a given project. It would be straightforward to build an MSBuild task that enumerates those assembly references, checking for associated source files to be added to the list of sources that get passed to nccs.