F# support - Githubissues

jkotas commented 7 years ago

Issue to track progress towards enabling native F# support.

The first step: try to compile F# "hello world" program using CoreRT and identify where the first blocking issue.

jkotas commented 7 years ago

tonerdo commented 7 years ago

First blocker: There's no support for MSBuild based F# projects with the dotnet cli. Is there an alternative to https://github.com/dotnet/corert/blob/master/Documentation/how-to-build-and-run-ilcompiler-in-console-shell-prompt.md#compiling-source-to-native-code-using-the-ilcompiler-you-built for building .NET Core apps with CoreRT?

jkotas commented 7 years ago

@tsolarin Thanks for being interested in F# on CoreRT! You can build the F# app using the current non-msbuild based .NET Core tooling, and then follow steps in https://github.com/dotnet/corert/blob/master/Documentation/how-to-build-and-run-ilcompiler-in-visual-studio-2015.md with modified c:\corert\bin\obj\Windows_NT.x64.Debug\ryujit.rsp:

Replace ...\repro.exe with path to your F# app (it should have .dll suffix)
Add -r:... for F# runtime .dlls

tonerdo commented 7 years ago

@jkotas I'm on OSX so unfortunately I couldn't use the method above. However, thanks to https://github.com/dotnet/netcorecli-fsc/tree/master/examples/preview3/console and a whole lot of tweaking on my end I was able to get MSBuild to successfully build F# on CoreCLR on OSX.

CoreRT

I went ahead to make a trial build on CoreRT. Below are the results

Using RyuJIT

dotnet build /t:LinkNative /p:IlcPath=/Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1

On first build attempt (possibly due to the lack of existing output files) IlcCompile throws the following exception:

Unhandled Exception: Internal.TypeSystem.TypeSystemException+FileNotFoundException: [TEMPORARY EXCEPTION MESSAGE] FileLoadErrorGeneric: FSharp.Core
            at ILCompiler.CompilerTypeSystemContext.GetModuleForSimpleName(String simpleName, Boolean throwIfNotFound)
            at ILCompiler.CompilerTypeSystemContext.ResolveAssembly(AssemblyName name, Boolean throwIfNotFound)
            at Internal.TypeSystem.Ecma.EcmaModule.ResolveAssemblyReference(AssemblyReferenceHandle handle)
            at Internal.TypeSystem.Ecma.EcmaModule.EcmaObjectLookupHashtable.CreateValueFromKey(EntityHandle handle)
            at Internal.TypeSystem.LockFreeReaderHashtable`2.CreateValueAndEnsureValueIsInTable(TKey key)
            at Internal.TypeSystem.LockFreeReaderHashtable`2.GetOrCreateValue(TKey key)
            at Internal.TypeSystem.Ecma.EcmaModule.GetObject(EntityHandle handle)
            at Internal.TypeSystem.Ecma.EcmaModule.ResolveTypeReference(TypeReferenceHandle handle)
            at Internal.TypeSystem.Ecma.EcmaModule.EcmaObjectLookupHashtable.CreateValueFromKey(EntityHandle handle)
            at Internal.TypeSystem.LockFreeReaderHashtable`2.CreateValueAndEnsureValueIsInTable(TKey key)
            at Internal.TypeSystem.LockFreeReaderHashtable`2.GetOrCreateValue(TKey key)
            at Internal.TypeSystem.Ecma.EcmaModule.GetObject(EntityHandle handle)
            at Internal.TypeSystem.Ecma.EcmaModule.ResolveMemberReference(MemberReferenceHandle handle)
            at Internal.TypeSystem.Ecma.EcmaModule.EcmaObjectLookupHashtable.CreateValueFromKey(EntityHandle handle)
            at Internal.TypeSystem.LockFreeReaderHashtable`2.CreateValueAndEnsureValueIsInTable(TKey key)
            at Internal.TypeSystem.LockFreeReaderHashtable`2.GetOrCreateValue(TKey key)
            at Internal.TypeSystem.Ecma.EcmaModule.GetObject(EntityHandle handle)
            at Internal.TypeSystem.Ecma.EcmaModule.GetMethod(EntityHandle handle)
            at ILCompiler.Metadata.Transform`1.HandleCustomAttributes(EcmaModule module, CustomAttributeHandleCollection attributes)
            at ILCompiler.Metadata.Transform`1.InitializeScopeDefinition(ModuleDesc module, ScopeDefinition scopeDefinition)
            at ILCompiler.Metadata.EntityMap`2.GetOrCreate[TConcreteEntity,TConcreteRecord](TConcreteEntity entity, Action`2 initializer)
            at ILCompiler.Metadata.Transform`1.HandleScopeDefinition(ModuleDesc module)
            at ILCompiler.Metadata.Transform`1.HandleNamespaceDefinition(ModuleDesc parentScope, String namespaceString)
            at ILCompiler.Metadata.Transform`1.InitializeTypeDef(MetadataType entity, TypeDefinition record)
            at ILCompiler.Metadata.EntityMap`2.Create[TConcreteEntity,TConcreteRecord](TConcreteEntity entity, Action`2 initializer)
            at ILCompiler.Metadata.Transform`1.HandleType(TypeDesc type)
            at ILCompiler.Metadata.MetadataTransform.Run[TPolicy](TPolicy policy, IEnumerable`1 modules)
            at ILCompiler.MetadataGeneration.EnsureMetadataGenerated()
            at ILCompiler.MetadataGeneration.GetMetadataBlob()
            at ILCompiler.DependencyAnalysis.MetadataNode.GetData(NodeFactory factory, Boolean relocsOnly)
            at ILCompiler.DependencyAnalysis.ObjectWriter.EmitObject(String objectFilePath, IEnumerable`1 nodes, NodeFactory factory)
            at ILCompiler.RyuJitCompilation.CompileInternal(String outputFile)
            at ILCompiler.Compilation.ILCompiler.ICompilation.Compile(String outputFile)
            at ILCompiler.Program.Run(String[] args)
            at ILCompiler.Program.Main(String[] args)
         /var/folders/tz/t9q0c6sd5xj5ktps1_1p9s400000gn/T/tmp5cd13436c8394d9b8f98d117df517263.exec.cmd: line 2:  9758 Abort trap: 6           "/Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/corerun" "/Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/ilc.exe" @"obj/Debug/netcoreapp1.0/native/ilc.rsp"

Subsequent build attempts produced a clang error viewable in the following build output:

Microsoft (R) Build Engine version 15.1.0.0
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 11/6/16 6:02:21 PM.
     1>Project "/Users/Toni/Workspace/FsCore/FsCore.fsproj" on node 1 (LinkNative;Build target(s)).
     1>CoreCompile:
       Skipping target "CoreCompile" because all output files are up-to-date with respect to the input files.
       IlcCompile:
       Skipping target "IlcCompile" because all output files are up-to-date with respect to the input files.
       LinkNative:
         clang obj/Debug/netcoreapp1.0/native/FsCore.o /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libbootstrapper.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libRuntime.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libSystem.Private.CoreLib.Native.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/framework/System.Native.a -o bin/Debug/netcoreapp1.0/native/FsCore -g -pthread -lstdc++ -ldl -lm
     1>clang : warning : argument unused during compilation: '-pthread' [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
         ld: file too small (length=0) file 'obj/Debug/netcoreapp1.0/native/FsCore.o' for architecture x86_64
     1>clang : error : linker command failed with exit code 1 (use -v to see invocation) [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>/Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/Microsoft.NETCore.Native.Unix.targets(67,5): error MSB3073: The command "clang obj/Debug/netcoreapp1.0/native/FsCore.o /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libbootstrapper.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libRuntime.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libSystem.Private.CoreLib.Native.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/framework/System.Native.a -o bin/Debug/netcoreapp1.0/native/FsCore -g -pthread -lstdc++ -ldl -lm" exited with code 1. [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>Done Building Project "/Users/Toni/Workspace/FsCore/FsCore.fsproj" (LinkNative;Build target(s)) -- FAILED.

Build FAILED.

Using CPP Code Generator

dotnet build /t:LinkNative /p:IlcPath=/Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1 /p:NativeCodeGen=cpp

The CppCodeGen approach achieves a successful build of an F# console project on CoreRT. Below is the build process output:

Microsoft (R) Build Engine version 15.1.0.0
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 11/6/16 6:08:06 PM.
     1>Project "/Users/Toni/Workspace/FsCore/FsCore.fsproj" on node 1 (LinkNative;Build target(s)).
     1>PrepareForBuild:
         Creating directory "bin/Debug/netcoreapp1.0/".
         Creating directory "obj/Debug/netcoreapp1.0/".
       CoreCompile:
         dotnet exec --depsfile /Users/Toni/.nuget/packages/.tools\dotnet-compile-fsc\1.0.0-preview2-020000\netcoreapp1.0\dotnet-compile-fsc.deps.json --additionalprobingpath /Users/Toni/.nuget/packages/ /Users/Toni/.nuget/packages/dotnet-compile-fsc\1.0.0-preview2-020000\lib\netcoreapp1.0\dotnet-compile-fsc.dll @/Users/Toni/Workspace/FsCore/obj/Debug/netcoreapp1.0/dotnet-compile.rsp
       IlcCompile:
         Creating directory "obj/Debug/netcoreapp1.0/native/".
         "/Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/corerun" "/Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/ilc.exe" @"obj/Debug/netcoreapp1.0/native/ilc.rsp"
       CppCompile:
         clang -I /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/inc obj/Debug/netcoreapp1.0/native/FsCore.cpp -o obj/Debug/netcoreapp1.0/native/FsCore.o -O0 -c -Wno-invalid-offsetof
     1>obj/Debug/netcoreapp1.0/native/FsCore.cpp(212205,19): warning GEF0C8401: comparison of unsigned expression < 0 is always false [-Wtautological-compare] [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
                 if ((uint32_t)_3 < (uint32_t)0) {
                     ~~~~~~~~~~~~ ^ ~~~~~~~~~~~
     1>obj/Debug/netcoreapp1.0/native/FsCore.cpp(212371,19): warning GEF0C8401: comparison of unsigned expression < 0 is always false [-Wtautological-compare] [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
                 if ((uint32_t)_3 < (uint32_t)0) {
                     ~~~~~~~~~~~~ ^ ~~~~~~~~~~~
         2 warnings generated.
       LinkNative:
         Creating directory "bin/Debug/netcoreapp1.0/native".
         clang obj/Debug/netcoreapp1.0/native/FsCore.o /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libbootstrappercpp.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libPortableRuntime.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libSystem.Private.CoreLib.Native.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/framework/System.Native.a -o bin/Debug/netcoreapp1.0/native/FsCore -g -pthread -lstdc++ -ldl -lm
     1>clang : warning : argument unused during compilation: '-pthread' [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
       CopyFilesToOutputDirectory:
         Copying file from "obj/Debug/netcoreapp1.0/FsCore.dll" to "bin/Debug/netcoreapp1.0/FsCore.dll".
         FsCore -> /Users/Toni/Workspace/FsCore/bin/Debug/netcoreapp1.0/FsCore.dll
         Copying file from "obj/Debug/netcoreapp1.0/FsCore.pdb" to "bin/Debug/netcoreapp1.0/FsCore.pdb".
     1>Done Building Project "/Users/Toni/Workspace/FsCore/FsCore.fsproj" (LinkNative;Build target(s)).

Build succeeded.

However, when I try to run the resulting native binary instead of the expected Hello World output I get libc++abi.dylib: terminating with uncaught exception of type unsigned int

Environment Details

dotnet --info output:

.NET Command Line Tools (1.0.0-preview3-004056)

Product Information:
 Version:            1.0.0-preview3-004056
 Commit SHA-1 hash:  ccc4968bc3

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.11
 OS Platform: Darwin
 RID:         osx.10.11-x64

jkotas commented 7 years ago

FileLoadErrorGeneric: FSharp.Core

This error is caused by FSharp.Core.dll missing on ilc command line. The project dependencies are not picked up automatically yet, Could you please manually specify FSharp.Core.dll for now - adding the following lines to your .fsproj file should do the trick:

<ItemGroup>
  <IlcReference Include="... path to FSharp.Code.dll ..." />
</ItemGroup>

Also, do not worry about CppCodeGen for now. It is behind RyuJIT codegen, so you would only find more problems with it; not less.

tonerdo commented 7 years ago

Thanks for that @jkotas I added that and tried another RyuJIT based build. Below is the build output:

Microsoft (R) Build Engine version 15.1.0.0
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 11/7/16 10:38:39 AM.
     1>Project "/Users/Toni/Workspace/FsCore/FsCore.fsproj" on node 1 (LinkNative;Build target(s)).
     1>CoreCompile:
       Skipping target "CoreCompile" because all output files are up-to-date with respect to the input files.
       IlcCompile:
       Skipping target "IlcCompile" because all output files are up-to-date with respect to the input files.
       LinkNative:
         clang obj/Debug/netcoreapp1.0/native/FsCore.o /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libbootstrapper.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libRuntime.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libSystem.Private.CoreLib.Native.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/framework/System.Native.a -o bin/Debug/netcoreapp1.0/native/FsCore -g -pthread -lstdc++ -ldl -lm
     1>clang : warning : argument unused during compilation: '-pthread' [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : object file (obj/Debug/netcoreapp1.0/native/FsCore.o) was built for newer OSX version (15.5) than being linked (10.11) [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : could not create compact unwind for _System_Private_CoreLib_System_Runtime_EH__UnhandledExceptionFailFastViaClasslib: does not use RBP or RSP based frame [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : could not create compact unwind for _System_Private_CoreLib_System_Globalization_FormatProvider_Number__Int32ToDecStr: does not use RBP or RSP based frame [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : could not create compact unwind for _System_Private_CoreLib_System_Globalization_FormatProvider_Number__UInt32ToDecStr: does not use RBP or RSP based frame [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : could not create compact unwind for _System_Private_CoreLib_System_Globalization_FormatProvider_Number__Int64ToDecStr: does not use RBP or RSP based frame [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : could not create compact unwind for _System_Private_CoreLib_System_Globalization_FormatProvider_Number__UInt64ToDecStr: does not use RBP or RSP based frame [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : could not create compact unwind for _System_Private_CoreLib_System_Globalization_FormatProvider_Number__FormatExponent: does not use RBP or RSP based frame [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : could not create compact unwind for _System_Private_CoreLib_System_Globalization_FormatProvider_Number__FormatFixed: does not use RBP or RSP based frame [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : could not create compact unwind for _System_Private_CoreLib_System_Globalization_FormatProvider_Number__DoubleToNumber: does not use RBP or RSP based frame [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : could not create compact unwind for _System_Diagnostics_Tracing_System_Diagnostics_Tracing_EventSource__WriteEvent_0: does not use RBP or RSP based frame [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : could not create compact unwind for _System_Console_Interop_Sys__StrError: does not use RBP or RSP based frame [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : could not create compact unwind for _System_Diagnostics_Tracing_System_Diagnostics_Tracing_EventSource__SendManifest: does not use RBP or RSP based frame [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : could not create compact unwind for _System_Private_CoreLib_System_String__IndexOfAny_1: does not use RBP or RSP based frame [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : could not create compact unwind for _System_Diagnostics_Tracing_System_Diagnostics_Tracing_EventSource__WriteMultiMergeInner: does not use RBP or RSP based frame [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : could not create compact unwind for _System_Private_CoreLib_Internal_Runtime_Augments_RuntimeAugments__NewMultiDimArray: does not use RBP or RSP based frame [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>ld : warning : can't parse dwarf compilation unit info in obj/Debug/netcoreapp1.0/native/FsCore.o [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
         Undefined symbols for architecture x86_64:
           "_EventWriteTransfer", referenced from:
               _System_Diagnostics_Tracing_System_Diagnostics_Tracing_EventSource__EventWriteTransferWrapper in FsCore.o
              (maybe you meant: _lsda0_System_Diagnostics_Tracing_System_Diagnostics_Tracing_EventSource__EventWriteTransferWrapper, _System_Diagnostics_Tracing_System_Diagnostics_Tracing_EventSource__EventWriteTransferWrapper )
         ld: symbol(s) not found for architecture x86_64
     1>clang : error : linker command failed with exit code 1 (use -v to see invocation) [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>/Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/Microsoft.NETCore.Native.Unix.targets(67,5): error MSB3073: The command "clang obj/Debug/netcoreapp1.0/native/FsCore.o /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libbootstrapper.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libRuntime.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/sdk/libSystem.Private.CoreLib.Native.a /Users/Toni/Workspace/corert/bin/Product/OSX.x64.Debug/packaging/publish1/framework/System.Native.a -o bin/Debug/netcoreapp1.0/native/FsCore -g -pthread -lstdc++ -ldl -lm" exited with code 1. [/Users/Toni/Workspace/FsCore/FsCore.fsproj]
     1>Done Building Project "/Users/Toni/Workspace/FsCore/FsCore.fsproj" (LinkNative;Build target(s)) -- FAILED.

Build FAILED.

jkotas commented 7 years ago

Undefined symbols for architecture x86_64: _EventWriteTransfer

This error is caused by CoreRT version of System.Diagnostics.Tracing not being ported to Unix yet. To workaround it for now, add a dummy throwing stub for EventWriteTransfer to https://github.com/dotnet/corert/blob/master/src/Native/Bootstrap/platform.unix.cpp

tonerdo commented 7 years ago

Awesome! Followed your directive and it built successfully. However, when I try to run the resulting native binary I get the following error:

Call to an unimplemented runtime method; execution cannot continue.
Method: GetNativeSystemInfo

tonerdo commented 7 years ago

/cc @jkotas

jkotas commented 7 years ago

As the message says, this means you are hitting unimplemented part of the system. The first step is to find out where it is coming from exactly: Start your program under lldb, set break on exceptions using break set -E C++, and look at the stacktrace where it is coming from.

I suspect that it will be coming from Environment.ProcessorCount. We have it ported to Unix in corefx repo already, so you just need to copy the implementation from there: https://github.com/dotnet/corefx/blob/master/src/System.Runtime.Extensions/src/System/Environment.Unix.cs#L420 and the related files (copy src\Common\src\Interop\Unix\System.Native\Interop.SysConf.cs; but not the C++ impl - we want to use that from corefx). It creates some duplication between repos, but it is not the first one - we will come back to it later.

Verify that it works, e.g. by running a simple program that does Console.WriteLine(Environment.ProcessorCount);, submit PR, and see where your F# test breaks next.

jkotas commented 7 years ago

@tsolarin Thank you for your help!

tonerdo commented 7 years ago

PR open. Had to port the C++ impl though, I couldn't achieve a successful build without it. My test F# program ("Hello World") was able to build and run without any errors

tonerdo commented 7 years ago

So we got Hello World working, what direction do we go next?

jkotas commented 7 years ago

Try more complex tests - I believe F# repo has a bunch of them.

tonerdo commented 7 years ago

Hey @jkotas it's me again 😀. I've been trying to build more complex F# programs with CoreRT and I've run into another error. Whenever I try to print formatted text (e.g. printf "%b" true) I get a FailFast: Runtime internal error exception. After a bit of debugging and digging through the visualfsharp source code I discovered that formatted text printing was implemented using reflection (especially Type.GetMethod().Invoke()). Further tests with C# and a look at this repo showed that reflection is not yet (fully?) supported in CoreRT.

My question: Do I help with porting required reflection apis from CoreCLR or would my time be better spent discovering/fixing other F# features that should work with the current state of CoreRT but don't?

jkotas commented 7 years ago

Runtime internal error

Where is this runtime internal error coming from? It looks like a more fundamental problem than just a missing reflection.

Type.GetMethod().Invoke()

The simple Type.GetMethod().Invoke() should work, as long as the methods are enabled for reflection. We have a very simple set of policies on what to enable reflection on. If you need to enable reflection for specific methods, you can list them in rd.xml file and then pass the path to the file using rdxml command line option. It should get you unblocked.

@MichalStrehovsky Anything you would add?

MichalStrehovsky commented 7 years ago

@MichalStrehovsky Anything you would add?

Yes, the rule is that if the method is statically reachable in the callgraph, and the type that owns the method was allocated (with a new or at least referenced with a typeof), it should be reflection-callable. If that's not the case, GetMethod will just give you null, as if the method wasn't there.

You can force the type and all of it's methods to be generated through RD.XML if the above conditions are not naturally met. We only support a limited subset of the RD.XML format.

Callstack at the time of failfast would help.

tonerdo commented 7 years ago

Hi @jkotas thanks for pointing out the rdxml workaround I'll give it a shot. @MichalStrehovsky here's the callstack at the time of failfast:

* thread #1: tid = 0x42c89, 0x000000010018edca FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl__getValueConverter + 1394, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x000000010018edca FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl__getValueConverter + 1394
    frame #1: 0x000000010011bdc8 FsCore`System_Private_CoreLib_System_Diagnostics_Debug__Assert + 12
    frame #2: 0x00000001001141f4 FsCore`System_Private_CoreLib_System_Array__GetRawSzArrayData + 28
    frame #3: 0x0000000100113082 FsCore`RhpStelemRef + 250
    frame #4: 0x000000010018edbd FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl__getValueConverter + 1381
    frame #5: 0x0000000100186c8c FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_PrintfBuilder_3<System_IO_System_IO_TextWriter__FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_Unit>__FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_PrintfBuilder_3__parseFromFormatSpecifier<> + 1152
    frame #6: 0x0000000100162b9e FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_PrintfBuilder_3<System_IO_System_IO_TextWriter__FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_Unit>__FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_PrintfBuilder_3__parseFormatString<> + 118
    frame #7: 0x000000010015b265 FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_PrintfBuilder_3<System_IO_System_IO_TextWriter__FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_Unit>__FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_PrintfBuilder_3__Build<FSharp_Core_Microsoft_FSharp_Core_FSharpFunc_2<FSharp_Core_Microsoft_FSharp_Core_FSharpOption_1<System_Private_CoreLib_System_Int32>__FSharp_Core_Microsoft_FSharp_Core_Unit>> + 89
    frame #8: 0x000000010014f38f FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl__cctor_1262_175<FSharp_Core_Microsoft_FSharp_Core_FSharpFunc_2<FSharp_Core_Microsoft_FSharp_Core_FSharpOption_1<System_Private_CoreLib_System_Int32>__FSharp_Core_Microsoft_FSharp_Core_Unit>__System_IO_System_IO_TextWriter__FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_Unit>__FSharp_Core_Microsoft_FSharp_Core_PrintfImpl__cctor_1262_175__Invoke<> + 39
    frame #9: 0x0000000100141f58 FsCore`System_Collections_Concurrent_System_Collections_Concurrent_ConcurrentDictionary_2<System_Private_CoreLib_System_String__System_Private_CoreLib_System_Tuple_2<FSharp_Core_Microsoft_FSharp_Core_FSharpFunc_2<FSharp_Core_Microsoft_FSharp_Core_FSharpFunc_2<FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_PrintfEnv_3<System_IO_System_IO_TextWriter__FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_Unit>>__FSharp_Core_Microsoft_FSharp_Core_FSharpFunc_2<FSharp_Core_Microsoft_FSharp_Core_FSharpOption_1<System_Private_CoreLib_System_Int32>__FSharp_Core_Microsoft_FSharp_Core_Unit>>__System_Private_CoreLib_System_Int32>>__System_Collections_Concurrent_System_Collections_Concurrent_ConcurrentDictionary_2__GetOrAdd<> + 100
    frame #10: 0x0000000100126b2c FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_Cache_4<FSharp_Core_Microsoft_FSharp_Core_FSharpFunc_2<FSharp_Core_Microsoft_FSharp_Core_FSharpOption_1<System_Private_CoreLib_System_Int32>__FSharp_Core_Microsoft_FSharp_Core_Unit>__System_IO_System_IO_TextWriter__FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_Unit>__FSharp_Core_Microsoft_FSharp_Core_PrintfImpl_Cache_4__Get<> + 104
    frame #11: 0x00000001001250f7 FsCore`FSharp_Core_Microsoft_FSharp_Core_PrintfModule__FSharp_Core_Microsoft_FSharp_Core_PrintfModule__PrintFormatToTextWriterThen<FSharp_Core_Microsoft_FSharp_Core_Unit__FSharp_Core_Microsoft_FSharp_Core_FSharpFunc_2<FSharp_Core_Microsoft_FSharp_Core_FSharpOption_1<System_Private_CoreLib_System_Int32>__FSharp_Core_Microsoft_FSharp_Core_Unit>> + 27
    frame #12: 0x000000010011877b FsCore`FsCore_Program__main + 67
    frame #13: 0x0000000100110f43 FsCore`FsCore__Module___StartupCodeMain + 87
    frame #14: 0x0000000100010a82 FsCore`main(argc=1, argv=0x00007fff5fbff3e0) + 242 at main.cpp:314
    frame #15: 0x00007fff8e5f85ad libdyld.dylib`start + 1

For further investigation here's the simple F# program I compiled:

[<EntryPoint>]
let main argv =
    let div x y =
        match y with
        | 0 -> None
        | _ -> Some(x/y)

    let res : int option = div 20 4
    printfn "Result: %A " res
    0 // return an integer exit code

ErikSchierboom commented 7 years ago

Just as a quick heads-up, I noticed that a design for how to handle tail calls in CoreRT has been merged recently. Progress being made!

wecing commented 7 years ago

Hi @tsolarin , may I ask how did you make the dotnet build command work? I built dotnet/cli and corert from source, and was able to compile .NET Core C# programs; but when I try to compile F# programs, I got:

The target "CoreCompile" does not exist in this project. [~/Code/tests/fsharp/fsharp.fsproj]
4 Warning(s)
1 Error(s)

Thanks!

tonerdo commented 7 years ago

@wecing here's a copy of my FsProj I use for building both on CoreCLR and CoreRT

<Project Sdk="FSharp.NET.Sdk;Microsoft.NET.Sdk">

  <Import Project="\Users\Toni\.nuget\packages\fsharp.net.sdk\1.0.0\build\FSharp.NET.Sdk.props" />
  <Import Project="$(MSBuildSDKsPath)\Microsoft.NET.Sdk\Sdk\Sdk.props" />

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp1.0</TargetFramework>
    <Version>1.0.0-alpha</Version>
    <EnableDefaultCompileItems>false</EnableDefaultCompileItems>
  </PropertyGroup>

  <ItemGroup>
    <Compile Include="Program.fs" />
    <EmbeddedResource Include="**\*.resx" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="FSharp.NET.Sdk" Version="1.0.0-beta-*" PrivateAssets="All" />
    <PackageReference Include="Microsoft.FSharp.Core.netcore" Version="1.0.0-alpha-161023" />
  </ItemGroup>

  <ItemGroup>
    <DotNetCliToolReference Include="dotnet-compile-fsc">
      <Version>1.0.0-preview2-020000</Version>
    </DotNetCliToolReference>
  </ItemGroup>

  <ItemGroup>
    <IlcReference Include="/Users/Toni/.nuget/packages/microsoft.fsharp.core.netcore/1.0.0-alpha-161023/lib/netstandard1.6/FSharp.Core.dll" />
  </ItemGroup>

  <Import Project="\Users\Toni\.nuget\packages\fsharp.net.sdk\1.0.0\build\FSharp.NET.Core.Sdk.targets" />
  <Import Project="$(MSBuildSDKsPath)\Microsoft.NET.Sdk\Sdk\Sdk.targets" />
  <Import Project="$(IlcPath)\Microsoft.NETCore.Native.targets" />

</Project>

You'd have to import FSharp.NET.Sdk.props and FSharp.NET.Sdk.targets from their nuget package installation location. Also, add an IlcReference to FSharp.Core.dll. That's how I fixed that error.

wecing commented 7 years ago

Hi @tsolarin , thanks for your answer. I copied your .fsproj and changed all occurences of your home directory to mine, but dotnet restore failed:

~/.nuget/packages/fsharp.net.sdk/1.0.0/build/FSharp.NET.Core.Sdk.targets(23,31): error MSB4022: The result "" of evaluating the value "$(FscTaskAssemblyPath)" of the "AssemblyFile" attribute in element <UsingTask> is not valid. [~/blah/blah.fsproj]

It is probably breaked by https://github.com/dotnet/netcorecli-fsc/commit/939248783c19112d51137be4c6b016446088dbc1 , which was checked in in early Jan.

So, @tsolarin , are you still able to build F# programs on CoreRT? (also cc @enricosada who submitted the patch)

enricosada commented 7 years ago

Hi everyone.

As a note, you are using old templates for fsproj /cc @wecing @tsolarin Updated can be obtained from .net core tools 1.0.1 (the rtm of dotnet core sdk) from running dotnet new console -lang f# (or lib).

More info in netcorecli-fsc wiki page

For previous dotnet/cli releases, see wiki and there are also templates in https://github.com/dotnet/netcorecli-fsc/tree/master/examples (call me if you get issues)

and example (generated by dotnet new console -lang f#):

<Project Sdk="FSharp.NET.Sdk;Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp1.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <Compile Include="Program.fs" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="FSharp.NET.Sdk" Version="1.0.*" PrivateAssets="All" />
    <PackageReference Include="FSharp.Core" Version="4.1.*" />
  </ItemGroup>

</Project>

In general, the diff with c# it's just:

extensions .fsproj
add FSharp.NET.Sdk; to Sdk attribute
add packagerefefence to FSharp.NET.Sdk sdk ( <PackageReference Include="FSharp.NET.Sdk" Version="1.0.*" PrivateAssets="All" /> )
add packagerefence to FSharp.Core ( <PackageReference Include="FSharp.Core" Version="4.1.*" /> )

IDE tooling is not mature yet, but from fsproj from cli and fsc compiler are ok afaik.

@jkotas @wecing @tsolarin @MichalStrehovsky any issue feel free to ping me directly anywhere (skype/twitter/github), or open an issue in https://github.com/dotnet/netcorecli-fsc I am really really interested in CoreRT

wecing commented 7 years ago

@tsolarin @enricosada Okay, sorry for spamming responses, but I fixed the issue by using FSharp.NET.Sdk/Sdk/Sdk.{props,targets} in $(MSBuildSDKsPath), instead of $HOME/.nuget. Here is my .fsproj:

<Project>
  <Import Project="$(MSBuildSDKsPath)\FSharp.NET.Sdk\Sdk\Sdk.props" />
  <Import Project="$(MSBuildSDKsPath)\Microsoft.NET.Sdk\Sdk\Sdk.props" />

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp1.0</TargetFramework>
    <Version>1.0.0-alpha</Version>
    <EnableDefaultCompileItems>false</EnableDefaultCompileItems>
  </PropertyGroup>

  <ItemGroup>
    <Compile Include="Program.fs" />
    <EmbeddedResource Include="**\*.resx" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="FSharp.NET.Sdk" Version="1.0.0-beta-*" PrivateAssets="All" />
    <PackageReference Include="Microsoft.FSharp.Core.netcore" Version="1.0.0-alpha-161023" />
  </ItemGroup>

  <ItemGroup>
    <DotNetCliToolReference Include="dotnet-compile-fsc">
      <Version>1.0.0-preview2-020000</Version>
    </DotNetCliToolReference>
  </ItemGroup>

  <ItemGroup>
    <IlcReference Include="/home/w/.nuget/packages/microsoft.fsharp.core.netcore/1.0.0-alpha-161023/lib/netstandard1.6/FSharp.Core.dll" />
  </ItemGroup>

  <Import Project="$(MSBuildSDKsPath)\FSharp.NET.Sdk\Sdk\Sdk.targets" />
  <Import Project="$(MSBuildSDKsPath)\Microsoft.NET.Sdk\Sdk\Sdk.targets" />
  <Import Project="$(IlcPath)\Microsoft.NETCore.Native.targets" />

</Project>

My dotnet command is built from source off dotnet/cli. It is okay to use @tsolarin 's approach and import .props and .targets from ~/.nuget, but you would also need to import ~/.nuget/packages/fsharp.compiler.tools/4.1.0/build/FSharp.Compiler.Tools.props.

What really confused me is that if you stick to @tsolarin 's ~/.nuget approach and import .props from fsharp.compiler.tools, dotnet would complain that fsharp.compiler.tools/.../Sdk.props is already imported. But compilation would succeed. If you use my .fsproj file I pasted above, it would not work if you remove the "PackageReference...FSharp.NET.Sdk" line. I am not very familiar with C#/F# toolchain so maybe I am missing something really obvious here.

enricosada commented 7 years ago

@wecing the import of .props/target is automatic from <PackageReference Include="FSharp.NET.Sdk". The issue is you are referencing an old version, 1.0.0-beta-*, try use 1.0.* like my previous example. And also because you are not using Sdk="FSharp.NET.Sdk.." attribute.

What version of dotnet/cli are you building? dotnet --version?

maybe this will do:

<Project Sdk="FSharp.NET.Sdk;Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp1.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <Compile Include="Program.fs" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="FSharp.NET.Sdk" Version="1.0.*" PrivateAssets="All" />
    <PackageReference Include="FSharp.Core" Version="4.1.*" />
  </ItemGroup>

  <ItemGroup>
    <IlcReference Include="$(NuGetPackageFolders)/fsharp.core/4.1.0/lib/netstandard1.6/FSharp.Core.dll" />
  </ItemGroup>

  <Import Project="$(IlcPath)\Microsoft.NETCore.Native.targets" />

</Project>

as a note, you can do dotnet build -v diag (it's the same as dotnet msbuild /t:Build /v:diag, see msbuild args) to see properties

wecing commented 7 years ago

@enricosada , my dotnet version is 2.0.0-preview1-005420 (built from source). Your .fsproj doesn't work, it would produce an error during dotnet build /t:LinkNative:

clang : error : no such file or directory: 'native/.o' [/home/w/Code/tests/fsharp/fs2/fs2.fsproj]
/home/w/Code/source/dotnet/corert/bin/Product/Linux.x64.Debug/packaging/publish1/Microsoft.NETCore.Native.targets(155,5): error MSB3073: The command "clang-3.9 native/.o -o native/ /home/w/Code/source/dotnet/corert/bin/Product/Linux.x64.Debug/packaging/publish1/sdk/libbootstrapper.a /home/w/Code/source/dotnet/corert/bin/Product/Linux.x64.Debug/packaging/publish1/sdk/libRuntime.a /home/w/Code/source/dotnet/corert/bin/Product/Linux.x64.Debug/packaging/publish1/sdk/libSystem.Private.CoreLib.Native.a /home/w/Code/source/dotnet/corert/bin/Product/Linux.x64.Debug/packaging/publish1/framework/System.Native.a /home/w/Code/source/dotnet/corert/bin/Product/Linux.x64.Debug/packaging/publish1/framework/libSystem.Globalization.Native.a -g -pthread -lstdc++ -ldl -lm -lrt" exited with code 1. [/home/w/Code/tests/fsharp/fs2/fs2.fsproj]
    0 Warning(s)
    2 Error(s)

But the FSharp.NET.Sdk/Sdk/Sdk.props file from $(MSBuildSDKsPath) is really simple:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

  <PropertyGroup>
    <MSBuildAllProjects>$(MSBuildAllProjects);$(MSBuildThisFileFullPath)</MSBuildAllProjects>
  </PropertyGroup>

</Project>

enricosada commented 7 years ago

I'll try to check this weekend. Strange use the dotnet/cli 2.0.0, an d not the rtm. But I don't know this repo

tonerdo commented 7 years ago

@wecing @enricosada I've taken a look and did the upgrade to use dotnet/cli 1.0.1 and here are my findings:

Using the template generated fsproj works ok when just building for CoreCLR, however some modifications are needed when trying to build for CoreRT. Here's my final layout as well as the steps I took to achieve a successful build:

Add lines 4 and 25 as specified in the CoreRT documentation If you don't you'd get the clang error @wecing is currently seeing clang : error : no such file or directory: 'native/.o'
Next you need to add lines 3 and 24 to get rid of the The target "CoreCompile" does not exist in the project. error. Replace the path in the image with yours
After this you'd have a successful build (you'd see a bunch of warnings, ignore them for now). However, when you try to run the resulting native binary you'd get a Abort trap: 6 error, that's where the IlcReference on line 21 comes in (see previous comments from @jkotas). Add that, build again and you'd get a runnable F# native binary 🎉
dotnet restore won't work unless you comment out lines 24 and 26, remember to uncomment them when you want to do a native build.

Try it with a simple "Hello World" #2165 app at first just to be sure. I'm currently working on making more complex F# programs work with CoreRT.

Here's a text version of the image for easy copy-paste

<Project Sdk="FSharp.NET.Sdk;Microsoft.NET.Sdk">

  <Import Project="\Users\Toni\.nuget\packages\FSharp.Net.Sdk\1.0.2\build\FSharp.NET.Sdk.props" />
  <Import Project="$(MSBuildSDKsPath)\Microsoft.NET.Sdk\Sdk\Sdk.props" />

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp1.1</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <Compile Include="Program.fs" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="FSharp.Core" Version="4.1.*" />
    <PackageReference Include="FSharp.NET.Sdk" Version="1.0.*" PrivateAssets="All" />
  </ItemGroup>

  <ItemGroup>
    <IlcReference Include="\Users\Toni\.nuget\packages\FSharp.Core\4.1.0.2\lib\netstandard1.6\FSharp.Core.dll" />
  </ItemGroup>

  <Import Project="\Users\Toni\.nuget\packages\FSharp.Net.Sdk\1.0.2\build\FSharp.NET.Core.Sdk.targets" />
  <Import Project="$(MSBuildSDKsPath)\Microsoft.NET.Sdk\Sdk\Sdk.targets" />
  <Import Project="$(IlcPath)\Microsoft.NETCore.Native.targets" />

</Project>

At this point building F# on CoreRT is not as smooth as the C# experience and the above workaround is needed as there's still a lot of work to be done

@wecing Let me know if you have any more questions.

wecing commented 7 years ago

@tsolarin @MichalStrehovsky I think the problem of printfn originates from GetMethod returning null on PrintfImpl.Specializations. GetMethod is invoked here. I am not quite sure how to whitelist it in rd.xml (tried but not successful), but it seems that all those Final* methods are only used through reflection, so they are not reachable from the callgraph.

tonerdo commented 7 years ago

@wecing that I already know. Calls to GetMethod return null when run on CoreRT, also when using C#. However, I'm still confused how the reflection tests pass without needing an rdxml and my own exact code still fails. That's what I'm trying to figure out.

wecing commented 7 years ago

@tsolarin If you comment out this line (so that GetHello is no longer statically reachable), that unit test will fail.

The PrintImpl.Specializations.Final* methods are only used through reflection, so they are not statically reachable. So if I understand it correctly CoreRT does not keep the metadata to be used by reflection in this case. One solution might be to manually invoke those Final* methods somewhere and rely on compiler optimization to remove the actual invocations so it becomes only a hint for the Compiler to keep these metadata, but that is probably not an elegant solution. @jkotas and @MichalStrehovsky what do you think? :)

MichalStrehovsky commented 7 years ago

So if I understand it correctly CoreRT does not keep the metadata to be used by reflection

We not only don't generate the metadata, but we also don't bother generating the code. The experience is pretty bare bones right now.

As @wecing mentioned, you either need to make sure the code path is statically reachable, or provide it as an additional compilation root (a place from which compilation starts compiling the static callgraph) externally with an RD.XML file.

The RD.XML will have something like this in it:

<?xml version="1.0" encoding="utf-8"?>
<Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata">
  <Application>
    <Assembly Name="repro">
      <Type Name="IFrobber" Dynamic="Required All" />
    </Assembly>
  </Application>
</Directives>

Assembly name is the name of the assembly that contains the definition of the type. The type name needs to be in the ECMA-335 SerString format (note it's a different format than the "real" RD.XML uses, and a bit more cumbersome). The compiler is pretty unfriendly if you don't get the exact string right, so it might require a bit of trial and error. (Useful piece of information is that generic type arguments need extra brackets, so it's Foo'1[[System.Object, System.Runtime]] if you want Foo<object>

Then you pass the XML file to ilc.exe as an additional parameter (--rdxml parameter).

charlesroddie commented 6 years ago

Can anyone put a test repo online? Could be just a hello world project. Then we can start testing to see what works and doesn't work.

charlesroddie commented 6 years ago

@zpodlovics The next project could be port the F# testsuite to CoreRT.

YES. Shall we set up a repo to test F#?

zpodlovics commented 6 years ago

@charlesroddie Good question. I think this needs coordination (to keep them in sync) with the F# team especially with @dsyme

The F# testsuite (as far as I know) are available here, but it will needs some fsproj / test runner script changes (some of them written in perl) to target CoreRT: https://github.com/Microsoft/visualfsharp/tree/master/tests/fsharp

dsyme commented 6 years ago

You can just take any of the source code in one of these subdirectories, e.g. start with this one, compile it and run it. In most cases it's just this for .NET Framework:

fsc.exe test.fsx
.\test.exe

so adjust that in the natural way. The test runner here runs various variations (debug on/off, optimize on/off etc).

If you are based on .NET SDK projects then you can also dump the contents of those test files into a .NET SDK project created following these instructions

dsyme commented 6 years ago

If you want to somehow add automated CoreRT testing to http://github.com/Microsoft/visualfsharp that would be fine I think. We already run Mono, .NET Framework and .NET Core.

It looks like all you have to add is

<PackageReference Include="Microsoft.DotNet.ILCompiler" Version="1.0.0-alpha-*" />

to the project file?

zpodlovics commented 6 years ago

@dsyme Basically yes, and a nuget.config also required because the ILCompiler is not yet available on nuget.org. I guess the nuget.config could be moved to the tests root directory. Using printf in AOT environment is usually fails, and fixing it is usually horrible (learned from earlier mono aot experience).

I have attached an example test project from tests/fsharp/core/array. Unfortunately it fails with

Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.FSharp.Core.PrintfImpl.PrintfBuilder`3.buildPlainFinal(Object[], Type[]) + 0xbd
   at test!<BaseAddress>+0x25ad65
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey, Func`2) + 0x83
   at test!<BaseAddress>+0x23b061
   at test!<BaseAddress>+0x260a26
   at Test.test_list_stableSortBy() + 0x56
   at <StartupCode$test>.$Test$fsx.main@() + 0x63
   at test!<BaseAddress>+0x2424d7

Aborted

Usage (Ubuntu 16.04, x86_64):

cd tests/fsharp/core/array
dotnet restore
dotnet publish -c Release -r linux-x64
./bin/Release/netcoreapp2.0/linux-x64/publish/test

tests_fsharp_core_array.zip

zpodlovics commented 6 years ago

I did a quick (and probably non-correct) search & replace with System.String.Format and System.Console.WriteLine, but sill fails with the following:

Unhandled Exception: System.PlatformNotSupportedException: Arrays with non-zero lower bounds are not supported.
   at test!<BaseAddress>+0x1b66e3
   at test!<BaseAddress>+0x250d67
   at test!<BaseAddress>+0x251150
   at Test.ArrayNonZeroBasedTestsSlice.runTest() + 0x41
   at <StartupCode$test>.$Test$fsx.main@() + 0x202
   at test!<BaseAddress>+0x2426b7

Aborted

Update (without zip update): commented out all multidimension array test modules, but still have the same exception:

GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /tmp/tests/fsharp/core/array/bin/Debug/netcoreapp2.0/linux-x64/publish/test...done.
(gdb) run
Starting program: /tmp/tests/fsharp/core/array/bin/Debug/netcoreapp2.0/linux-x64/publish/test 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7f1e402f1700 (LWP 2356)]
test1 = True
test2 = True
test3 = True
test4 = True
test5 = True
test6 = True
Unhandled Exception: System.PlatformNotSupportedException: Arrays with non-zero lower bounds are not supported.
   at test!<BaseAddress>+0x1ab78b
   at <StartupCode$test>.$Test$fsx.main@() + 0x7973
   at test!<BaseAddress>+0x236a3f

Thread 1 "test" received signal SIGABRT, Aborted.
0x00007f1e5cc30428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
54  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) where
#0  0x00007f1e5cc30428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007f1e5cc3202a in __GI_abort () at abort.c:89
#2  0x0000000000459656 in RaiseFailFastException (arg1=0x930, arg2=0x930, arg3=6) at /root/corert_1666639/src/Native/Runtime/unix/PalRedhawkUnix.cpp:174
#3  0x00000000005a1a9f in S_P_CoreLib_System_Runtime_EH__UnhandledExceptionFailFastViaClasslib ()
#4  0x00000000005a1fa5 in S_P_CoreLib_System_Runtime_EH__DispatchEx ()
#5  0x00000000005a1dda in S_P_CoreLib_System_Runtime_EH__RhThrowEx ()
#6  0x0000000000465135 in RhpThrowEx () at /root/corert_1666639/src/Native/Runtime/amd64/ExceptionHandling.S:153
#7  0x00000000005ab78c in S_P_CoreLib_Internal_Runtime_Augments_RuntimeAugments__NewMultiDimArray ()
#8  0x00000000004cd2f4 in test__StartupCode_test___Test_fsx__main_ ()
#9  0x0000000000636a40 in test__Module___StartupCodeMain () at <stdin>:16707566
#10 0x0000000000410b05 in main (argc=1, argv=0x7ffe3354b4b8) at /root/corert_1666639/src/Native/Bootstrap/main.cpp:372

tests_fsharp_core_array_without_printf.zip

Update2: after disabling bug6447 it runs, but most 2d/3d/4d tests are disabled due the non-zero lower bound missing support.

tests_fsharp_core_array_without_printf_without_multidim.zip

charlesroddie commented 6 years ago

CoreRT/.Net Native has no plans to implement non-zero based arrays across all platforms. That shouldn't be a problem. Are any tests failing with this error that you wouldn't expect to fail? I.e. any tests that don't explicity ask for non-zero-based arrays?

dotnet / corert

F# support #2057

CoreRT

Using RyuJIT

Using CPP Code Generator

Environment Details