Be able to build on Mono

[ErikSchierboom]

Currently, there is only a build script for Windows. There should also be a build script that enables it to build on Mono.

[jtattermusch]

+1, installing the nuget package breaks our build on mono

related issue in google/protobuf repository: https://github.com/google/protobuf/issues/1054

[clairernovotny]

There are two aspects to this. One is running the tool on mono via the targets. That should be easy with the right targets checks/conditions -- if (unix) mono + cmd ...

The other part is getting the tool to have the data it needs on mono/Linux. For "classic PCLs", it needs the reference assemblies from the C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETPortable\{framework}\Profile\{profile} directory to know what package versions to use. Does that exist on Linux/mono?

For projects that use .NET Core packages (the new ones from NuGet), then those work just fine on Linux/Mono because it doesn't need to read the assembly versions from disk.

Another option is to have a conditional that disables the target so it can be run on windows but doesn't fail a Linux build.

What do you all think is the best option here?

[jtattermusch]

On mono, the way how to get access to the PCL libraries is to install the referenceassemblies-pcl package from mono distribution (see http://www.mono-project.com/docs/getting-started/install/linux/).

Listing contents of that package:

...
/usr/lib/mono/xbuild-frameworks/.NETPortable/v4.5/Profile/Profile259
/usr/lib/mono/xbuild-frameworks/.NETPortable/v4.5/Profile/Profile259/System.Diagnostics.Contracts.dll
/usr/lib/mono/xbuild-frameworks/.NETPortable/v4.5/Profile/Profile259/System.IO.dll
/usr/lib/mono/xbuild-frameworks/.NETPortable/v4.5/Profile/Profile259/SupportedFrameworks
/usr/lib/mono/xbuild-frameworks/.NETPortable/v4.5/Profile/Profile259/SupportedFrameworks/Windows 8.xml
/usr/lib/mono/xbuild-frameworks/.NETPortable/v4.5/Profile/Profile259/SupportedFrameworks/.NET Framework 4.5.xml
/usr/lib/mono/xbuild-frameworks/.NETPortable/v4.5/Profile/Profile259/SupportedFrameworks/Windows Phone Silverlight 8.xml
/usr/lib/mono/xbuild-frameworks/.NETPortable/v4.5/Profile/Profile259/SupportedFrameworks/Windows Phone 8.1.xml
/usr/lib/mono/xbuild-frameworks/.NETPortable/v4.5/Profile/Profile259/System.Runtime.Serialization.Primitives.dll
/usr/lib/mono/xbuild-frameworks/.NETPortable/v4.5/Profile/Profile259/System.Text.Encoding.dll
/usr/lib/mono/xbuild-frameworks/.NETPortable/v4.5/Profile/Profile259/System.Xml.XDocument.dll
...
...
...

[jtattermusch]

One possible workaround would also be to detect the build is not running on Windows and skip the regenerate build step without failing the build (and perhaps printing a warning message). For our use case, we can easily generate the dependencies on Windows and not breaking the build on Mono would be all we need.

[clairernovotny]

@jtattermusch The regenerate step could work in some instances, just not others. Is there a way to detect via an env variable if that package is installed and its location?

Is there a place to see the full list of contents of that referenceassemblies-pcl package? We'd want to make sure it has all of them and not just 259.

In the end there are two possibilities:

1. A runtime detection within RefGen that it's on Unix and should simply warn and exit if it's in a place where it needs but does not have the reference assemblies
2. Just not run refgen at all on Unix and warn from the msbuild script

[clairernovotny]

One minor thing -- I don't really have a unix/mono environment to build on. I'm happy to make changes based on variables, but I would need someone's help in validating that the changes work as expected.

[clairernovotny]

@jtattermusch can you please test out the version from the CI feed with the unix support? https://ci.appveyor.com/project/onovotny/referencegenerator/build/1.4.1-unix-support.1+5%20(Build%2022)/artifacts

[jtattermusch]

I tested it and there is still some problem:

Executing: mono "/Users/jtattermusch/github/protobuf/csharp/src/packages/NuSpec.ReferenceGenerator.1.4.1-unix-support01/build/portable-net45+win+wpa81+wp80+MonoAndroid10+xamarinios10+MonoTouch10/../../tools/RefGen.exe" ";" dotnet "/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/Google.Protobuf.nuspec;/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/Google.Protobuf.nuspec" "/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/" "/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/bin/Release//Google.Protobuf.dll"
                error RG001: Access to the path '/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/' is denied.
/Users/jtattermusch/github/protobuf/csharp/src/packages/NuSpec.ReferenceGenerator.1.4.1-unix-support01/build/portable-net45+win+wpa81+wp80+MonoAndroid10+xamarinios10+MonoTouch10/NuSpec.ReferenceGenerator.targets: error : Command 'mono "/Users/jtattermusch/github/protobuf/csharp/src/packages/NuSpec.ReferenceGenerator.1.4.1-unix-support01/build/portable-net45+win+wpa81+wp80+MonoAndroid10+xamarinios10+MonoTouch10/../../tools/RefGen.exe" ";" dotnet "/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/Google.Protobuf.nuspec;/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/Google.Protobuf.nuspec" "/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/" "/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/bin/Release//Google.Protobuf.dll"' exited with code: 255.

When trying to execute the tool directly using the command above:

$ mono "/Users/jtattermusch/github/protobuf/csharp/src/packages/NuSpec.ReferenceGenerator.1.4.1-unix-support01/build/portable-net45+win+wpa81+wp80+MonoAndroid10+xamarinios10+MonoTouch10/../../tools/RefGen.exe"
error RG001: Index was outside the bounds of the array.

In case it matters, please note that I am trying this on a MacOS machine with Xamarin and mono installed. (that was the easiest environment for me to try it with).

[jtattermusch]

Please ignore the "Index was outside the bounds" error. That's because I didn't supply the right arguments. If I run it with the right args, it really gives the "access denied" error:

$ mono "/Users/jtattermusch/github/protobuf/csharp/src/packages/NuSpec.ReferenceGenerator.1.4.1-unix-support01/build/portable-net45+win+wpa81+wp80+MonoAndroid10+xamarinios10+MonoTouch10/../../tools/RefGen.exe" ";" dotnet "/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/Google.Protobuf.nuspec;/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/Google.Protobuf.nuspec" "/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/" "/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/bin/Release//Google.Protobuf.dll"
error RG001: Access to the path '/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/' is denied.

[clairernovotny]

Ok -- is that a valid scenario though, the access denied or is that really an error?

Is this a case of the file permissions being wrong or something not working within mono?

[jtattermusch]

I think that's an actual problem somewhere in RefGen.exe. The directory is definitely accessible on my machine. What exactly are you doing with it in RefGen?

[clairernovotny]

Something's strange then...I'm simply calling save on an XDocument: https://github.com/onovotny/ReferenceGenerator/blob/features/unix-support/src/ReferenceGenerator/Program.cs#L176

[jtattermusch]

My guess is that you are actually trying to read/write from the directory as if it was a regular file. Perhaps one-off error with cmdline args or a typo in arg list?

[clairernovotny]

Is there any chance you can step through this on the debugger? You should be able to paste in the command you were doing manually as the startup cmd / parameters. The program is pretty linear and it should become obvious.

The arg list issue might be as well as it's being invoked from mono - perhaps something's not translating well? A debugger would tell for sure.

[jtattermusch]

The args should be as follows:

// args 0: NuGetTargetMonikers -- .NETPlatform,Version=v5.0  
            // args 1: TFM's to generate, semi-colon joined. E.g.: dotnet;uap10.0 
            // args 2: nuspec file
            // args 3: project file (csproj/vbproj, etc). Used to look for packages.config/project.json and references. should match order of target files
            // args 4: target files, semi-colon joined

The actual params you are invoking it with are as follows: ";" dotnet "/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/Google.Protobuf.nuspec;/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/Google.Protobuf.nuspec" "/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/" "/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/bin/Release//Google.Protobuf.dll" error RG001: Access to the path '/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/' is denied.

So it looks like "/Users/jtattermusch/github/protobuf/csharp/src/Google.Protobuf/" should refer to the .csproj file instead. Trying to open a directory as if it was a .csproj file probably causes the "access denied" exception - which makes sense.

[clairernovotny]

I literally copy/pasted the command in the targets from the regular one and prepended mono to it. Are the rest of the data there but off by 1?

[jtattermusch]

Perhaps "@(NuSpecProjectFile)" has a wrong value?

[jtattermusch]

Anyway, if you are going to fail on Unix anyway, you might as well fail early and the effect is the same.

[clairernovotny]

Gitter? https://gitter.im/onovotny/ReferenceGenerator

[clairernovotny]

I'd like it to work on unix

[clairernovotny]

As an interim workaround until full unix support can be tested, I've disabled running refgen in the targets.

This package should have that in it: https://ci.appveyor.com/project/onovotny/referencegenerator/build/1.4.1-unix-support.1+9%20(Build%2029)/artifacts

If you can verify, I can push that out to NuGet.

Thanks!

[jtattermusch]

I tested the newest nuget package and I can confirm that our mono build works fine when using it. Can we get a new version of referenceGenerator uploaded to nuget.org soon? Thanks for your work!

[clairernovotny]

1.4.1 should be live on nuget.org now.

This still needs investigating to actually run on mono.