search

fsprojects / IfSharp

F# for Jupyter Notebooks
Other
442 stars 71 forks source link

Low level failure when calculations get large on Mono #143

Closed npmurphy closed 6 years ago

npmurphy commented 7 years ago

Description

Cells sometimes silently fail to execute variable assignments. Errors are logged in shell.log

Repro steps

  1. Unzip and run the attached notebook file.

Expected behavior

All the code in this notebook should run without error.

Actual behavior

Code for the first example fails to return its results to the IfSharp notebook.

Known workarounds

Related information

This problem has existed in IfSharp at least version 1.0 on both OSx and Linux. Problem code works fine in the fsharpi.exe environment. Problem code runs fine on Windows with .net (not tested with Mono on Windows).

problem book.ipynb.zip

cgravill commented 7 years ago

I see the same on Mac, cells for ease of inspections:

let rng = System.Random()
let n = 100

(note: change n to see it working)

let str_array = Array.init n 
                    (fun _ -> Array.init n (fun _ -> rng.NextDouble() |> sprintf "%f") |> String.concat "\t")
let sigbdata = str_array |> Array.map (fun (l:string) -> (l.Split [|'\t'|]) |> Array.map float)
sigbdata

Shell.log for reference, if you use small values it completes but large values fail

2017-04-18 16:46:30 - let n = 50
2017-04-18 16:46:30 - let str_array = Array.init n 
2017-04-18 16:46:30 -                     (fun _ -> Array.init n (fun _ -> rng.NextDouble() |> sprintf "%f") |> String.concat "\t")
2017-04-18 16:46:30 - let sigbdata = str_array |> Array.map (fun (l:string) -> (l.Split [|'\t'|]) |> Array.map float)
2017-04-18 16:46:30 - ExceptionOperation failed. The error text has been printed in the error stream. To return the corresponding FSharpErrorInfo use the EvalInteractionNonThrowing, EvalScriptNonThrowing or EvalExpressionNonThrowing
2017-04-18 16:46:30 -   at Microsoft.FSharp.Compiler.Interactive.Shell+FsiEvaluationSession.commitResult[a,b] (Microsoft.FSharp.Core.FSharpChoice`2[T1,T2] res) [0x00030] in <58823331ddab8ea7a745038331338258>:0 
2017-04-18 16:46:30 -   at Microsoft.FSharp.Compiler.Interactive.Shell+FsiEvaluationSession.EvalInteraction (System.String code) [0x0001a] in <58823331ddab8ea7a745038331338258>:0 
2017-04-18 16:46:30 -   at IfSharp.Kernel.IfSharpKernel.preprocessAndEval (System.String code) [0x001dc] in <58f63047f8575471a74503834730f658>:0 
2017-04-18 16:46:30 -   at IfSharp.Kernel.IfSharpKernel.executeRequest (IfSharp.Kernel.KernelMessage msg, IfSharp.Kernel.ExecuteRequest content) [0x00094] in <58f63047f8575471a74503834730f658>:0 
2017-04-18 16:46:31 - sigbdata
2017-04-18 16:46:38 - let n = 5
2017-04-18 16:46:38 - let str_array = Array.init n 
2017-04-18 16:46:38 -                     (fun _ -> Array.init n (fun _ -> rng.NextDouble() |> sprintf "%f") |> String.concat "\t")
2017-04-18 16:46:38 - let sigbdata = str_array |> Array.map (fun (l:string) -> (l.Split [|'\t'|]) |> Array.map float)
2017-04-18 16:46:39 - sigbdata

Mono details:

Mono JIT compiler version 4.8.1 (mono-4.8.0-branch/22a39d7 Fri Apr  7 12:00:08 EDT 2017)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
    TLS:           normal
    SIGSEGV:       altstack
    Notification:  kqueue
    Architecture:  x86
    Disabled:      none
    Misc:          softdebug 
    LLVM:          yes(3.6.0svn-mono-master/8b1520c)
    GC:            sgen

Mac:

ProductName:    Mac OS X
ProductVersion: 10.12.3
BuildVersion:   16D32

I don't generally run on Mono, do you have any ideas how to proceed @sylvanc ? I can probably make this more fault evident rather than silently failing.

cgravill commented 7 years ago

Switching to EvalInteractionNonThrowing didn't clarify what's happening either

2017-04-19 11:18:14 - NullReferenceExceptionObject reference not set to an instance of an object
2017-04-19 11:18:14 -   at Microsoft.FSharp.Core.Operators.FailurePattern (System.Exception error) [0x00001] in <567375de904cf4daa7450383de757356>:0 
2017-04-19 11:18:14 -   at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ReraiseIfWatsonable (System.Exception exn) [0x0002d] in <58823331ddab8ea7a745038331338258>:0 
2017-04-19 11:18:14 -   at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.ErrorRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x00087] in <58823331ddab8ea7a745038331338258>:0 
2017-04-19 11:18:14 -   at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ReraiseIfWatsonable (System.Exception exn) [0x00040] in <58823331ddab8ea7a745038331338258>:0 
2017-04-19 11:18:14 -   at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.ErrorRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x00087] in <58823331ddab8ea7a745038331338258>:0 
2017-04-19 11:18:14 -   at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ReraiseIfWatsonable (System.Exception exn) [0x00040] in <58823331ddab8ea7a745038331338258>:0 
2017-04-19 11:18:14 -   at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.ErrorRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x00087] in <58823331ddab8ea7a745038331338258>:0
cgravill commented 7 years ago

I have hardened the expression evaluation so that errors in the compiler services are also caught and printed:

screen shot 2017-04-19 at 15 49 25

This doesn't address the actual issue but at least provides more information rather than failing silently.

cgravill commented 7 years ago

For the error itself, there seem to be a few other similar cases:

http://stackoverflow.com/questions/40350362/memory-exception-from-math-net-when-generating-100-x-100-data-points https://github.com/fsharp/FSharp.Data/issues/1038

cgravill commented 7 years ago

I've put together a test case as a console application which does a similar set-up and calls EvalInteractionNonThrowing. This runs to completion without the above issues. Trying to include more of what we're doing next.

open System
open System.IO
open System.Text
open Microsoft.FSharp.Compiler.SourceCodeServices
open Microsoft.FSharp.Compiler.Interactive.Shell

//manual reference to: \packages\FSharp.Compiler.Tools.4.1.15\tools\FSharp.Compiler.Interactive.Settings.dll

/// Extend the `fsi` object with `fsi.AddHtmlPrinter` 
let addHtmlPrinter = """
    module FsInteractiveService = 
        let mutable htmlPrinters = new ResizeArray<System.Type * (obj -> seq<string * string> * string)>()
        let htmlPrinterParams = System.Collections.Generic.Dictionary<string, obj>()
        do htmlPrinterParams.["html-standalone-output"] <- false

    type Microsoft.FSharp.Compiler.Interactive.InteractiveSession with
        member x.HtmlPrinterParameters = FsInteractiveService.htmlPrinterParams
        member x.AddHtmlPrinter<'T>(f:'T -> seq<string * string> * string) = 
            FsInteractiveService.htmlPrinters.Add(typeof<'T>, fun (value:obj) ->
                f (value :?> 'T))"""

let internal startSession () =
    let sbOut = new StringBuilder()
    let sbErr = new StringBuilder()
    let sbPrint = new StringBuilder()
    let inStream = new StringReader("")
    let outStream = new StringWriter(sbOut)
    let errStream = new StringWriter(sbErr)
    let printStream = new StringWriter(sbPrint)

    let fsiObj = Microsoft.FSharp.Compiler.Interactive.Settings.fsi
    let fsiConfig = FsiEvaluationSession.GetDefaultConfiguration(fsiObj, false)
    let args = [|"--noninteractive"; "--define:HAS_FSI_ADDHTMLPRINTER" |]
    let fsiSession = FsiEvaluationSession.Create(fsiConfig, args, inStream, outStream, errStream)

    // Load the `fsi` object from the right location of the `FSharp.Compiler.Interactive.Settings.dll`
    // assembly and add the `fsi.AddHtmlPrinter` extension method; then clean it from FSI output
    let origLength = sbOut.Length
    let fsiLocation = typeof<Microsoft.FSharp.Compiler.Interactive.InteractiveSession>.Assembly.Location    
    fsiSession.EvalInteraction("#r @\"" + fsiLocation + "\"")
    fsiSession.EvalInteraction(addHtmlPrinter)
    sbOut.Remove(origLength, sbOut.Length-origLength) |> ignore

    // Get reference to the extra HTML printers registered inside the FSI session
    let extraPrinters = 
        unbox<ResizeArray<System.Type * (obj -> seq<string * string> * string)>>
          (fsiSession.EvalExpression("FsInteractiveService.htmlPrinters").Value.ReflectionValue)

    //Console.SetOut(printStream)
    sbErr, sbOut, sbPrint, extraPrinters, fsiSession

[<EntryPoint>]
let main argv =
    let sbErr, sbOut, sbPrint, extraPrinters, fsiEval = startSession()

    let newCode = """let rng = System.Random()
let n = 100
let str_array =
    Array.init n 
                (fun _ -> Array.init n (fun _ -> rng.NextDouble() |> sprintf "%f") |> String.concat "\t")
let sigbdata = str_array |> Array.map (fun (l:string) -> (l.Split [|'\t'|]) |> Array.map float)
sigbdata"""

    try
        let value, errors = fsiEval.EvalInteractionNonThrowing newCode

        printfn "Expression evaluation successful: %A" value

        match value with
        | Choice1Of2 value ->
            printfn "Success %A" sbOut
        | Choice2Of2 exn ->
            printfn "Failure %A" exn

    with exn ->
        printfn "Expression evaluation failed: %s %s %s" exn.Message Environment.NewLine exn.StackTrace

    printfn "%A" argv
    0 // return an integer exit code
cgravill commented 7 years ago

Same behaviour after update to Mono JIT compiler version 5.0.1.1 on OS X.

Expression evaluation failed: Object reference not set to an instance of an object
NullReferenceExceptionObject reference not set to an instance of an object
  at Microsoft.FSharp.Core.Operators.FailurePattern (System.Exception error) [0x00001] in <5939249c904cf4daa74503839c243959>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ReraiseIfWatsonable (System.Exception exn) [0x0002d] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.ErrorRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000d2] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.StopProcessingRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000b5] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.InteractiveCatch[b] (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger errorLogger, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] f, b istate) [0x00033] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.ExecInteraction (Microsoft.FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, Microsoft.FSharp.Compiler.CompileOps+TcConfig tcConfig, Microsoft.FSharp.Compiler.Interactive.Shell+FsiDynamicCompilerState istate, Microsoft.FSharp.Compiler.Ast+ParsedFsiInteraction action, Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger errorLogger) [0x00010] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.execParsedInteractions (Microsoft.FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, Microsoft.FSharp.Compiler.CompileOps+TcConfig tcConfig, Microsoft.FSharp.Compiler.Interactive.Shell+FsiDynamicCompilerState istate, Microsoft.FSharp.Core.FSharpOption`1[T] action, Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger errorLogger, Microsoft.FSharp.Core.FSharpOption`1[T] lastResult) [0x002a4] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+clo@2089-298.Invoke (Microsoft.FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, Microsoft.FSharp.Compiler.CompileOps+TcConfig tcConfig, Microsoft.FSharp.Compiler.Interactive.Shell+FsiDynamicCompilerState istate) [0x00003] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Core.FSharpFunc`2[T,TResult].InvokeFast[V,W] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] func, T arg1, TResult arg2, V arg3) [0x0000d] in <5939249c904cf4daa74503839c243959>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.mainThreadProcessAction[a,b] (a ctok, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] action, b istate) [0x0004a] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ReraiseIfWatsonable (System.Exception exn) [0x00040] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.ErrorRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000d2] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.StopProcessingRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000b5] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.mainThreadProcessAction[a,b] (a ctok, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] action, b istate) [0x00104] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.mainThreadProcessParsedInteractions (Microsoft.FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger errorLogger, Microsoft.FSharp.Core.FSharpOption`1[T] action, Microsoft.FSharp.Compiler.Interactive.Shell+FsiDynamicCompilerState istate) [0x0000b] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+EvalInteraction@2232.Invoke (Microsoft.FSharp.Compiler.Interactive.Shell+FsiDynamicCompilerState istate) [0x00013] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.InteractiveCatch[b] (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger errorLogger, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] f, b istate) [0x00018] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ReraiseIfWatsonable (System.Exception exn) [0x00040] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.ErrorRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000d2] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.StopProcessingRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000b5] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.InteractiveCatch[b] (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger errorLogger, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] f, b istate) [0x00033] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.EvalInteraction (Microsoft.FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, System.String sourceText, System.String scriptFileName, Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger errorLogger) [0x00058] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiEvaluationSession.EvalInteractionNonThrowing (System.String code) [0x0001c] in <590496d0ddab8ea7a7450383d0960459>:0 
  at IfSharp.Kernel.IfSharpKernel.executeRequest (IfSharp.Kernel.KernelMessage msg, IfSharp.Kernel.ExecuteRequest content) [0x00096] in <5943b5ddf8575471a7450383ddb54359>:0
dsyme commented 7 years ago

@cgravill Since this is a Mono bug, can you build a repro using either

  1. compiled code (fsharpc), or
  2. just using the "fsharpi" command line tool?
cgravill commented 7 years ago

@dsyme I've not been able to yet, there's some code above (https://github.com/fsprojects/IfSharp/issues/143#issuecomment-297969126) which performs equivalent operations as a command-line tool written against the compiler services. This successfully runs to completion without issue on my OS X machine.

I'm guessing we have somehow have a slightly different library via redirects or the runtime context called by Jupyter is different triggering the bug. I don't have a lot of experience debugging these things on OS X so it's going pretty slowly, expert volunteers on Mono/OS X/ Linux are most welcome!

xperiandri commented 7 years ago

Is this about the same?

Expression evaluation failed: Object reference not set to an instance of an object
NullReferenceExceptionObject reference not set to an instance of an object
  at Microsoft.FSharp.Core.Operators.FailurePattern (System.Exception error) [0x00001] in <5893d081904cf4daa745038381d09358>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ReraiseIfWatsonable (System.Exception exn) [0x0002d] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.ErrorRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000d2] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.StopProcessingRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000b5] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.InteractiveCatch[b] (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger errorLogger, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] f, b istate) [0x00033] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.ExecInteraction (Microsoft.FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, Microsoft.FSharp.Compiler.CompileOps+TcConfig tcConfig, Microsoft.FSharp.Compiler.Interactive.Shell+FsiDynamicCompilerState istate, Microsoft.FSharp.Compiler.Ast+ParsedFsiInteraction action, Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger errorLogger) [0x00010] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.execParsedInteractions (Microsoft.FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, Microsoft.FSharp.Compiler.CompileOps+TcConfig tcConfig, Microsoft.FSharp.Compiler.Interactive.Shell+FsiDynamicCompilerState istate, Microsoft.FSharp.Core.FSharpOption`1[T] action, Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger errorLogger, Microsoft.FSharp.Core.FSharpOption`1[T] lastResult) [0x002a4] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+clo@2089-298.Invoke (Microsoft.FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, Microsoft.FSharp.Compiler.CompileOps+TcConfig tcConfig, Microsoft.FSharp.Compiler.Interactive.Shell+FsiDynamicCompilerState istate) [0x00003] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Core.FSharpFunc`2[T,TResult].InvokeFast[V,W] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] func, T arg1, TResult arg2, V arg3) [0x0000d] in <5893d081904cf4daa745038381d09358>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.mainThreadProcessAction[a,b] (a ctok, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] action, b istate) [0x0004a] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ReraiseIfWatsonable (System.Exception exn) [0x00040] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.ErrorRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000d2] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.StopProcessingRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000b5] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.mainThreadProcessAction[a,b] (a ctok, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] action, b istate) [0x00104] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.mainThreadProcessParsedInteractions (Microsoft.FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger errorLogger, Microsoft.FSharp.Core.FSharpOption`1[T] action, Microsoft.FSharp.Compiler.Interactive.Shell+FsiDynamicCompilerState istate) [0x0000b] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+EvalInteraction@2232.Invoke (Microsoft.FSharp.Compiler.Interactive.Shell+FsiDynamicCompilerState istate) [0x00013] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.InteractiveCatch[b] (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger errorLogger, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] f, b istate) [0x00018] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ReraiseIfWatsonable (System.Exception exn) [0x00040] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.ErrorRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000d2] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.StopProcessingRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000b5] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.InteractiveCatch[b] (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger errorLogger, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] f, b istate) [0x00033] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiInteractionProcessor.EvalInteraction (Microsoft.FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, System.String sourceText, System.String scriptFileName, Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger errorLogger) [0x00058] in <590496d0ddab8ea7a7450383d0960459>:0 
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiEvaluationSession.EvalInteractionNonThrowing (System.String code) [0x0001c] in <590496d0ddab8ea7a7450383d0960459>:0 
  at IfSharp.Kernel.IfSharpKernel.executeRequest (IfSharp.Kernel.KernelMessage msg, IfSharp.Kernel.ExecuteRequest content) [0x00096] in <599dca641b085d3da745038364ca9d59>:0 
#load "Paket.fsx"
Paket.Package
  [ "FsAlg" ]
#load "Paket.Generated.Refs.fsx"

open FsAlg.Generic
open System

let m = 9
let n = 100
let rand = Random()
let w0 = rand.NextDouble()
let w1 = rand.NextDouble()
let gam = 0.01;

let ERS = Array2D.zeroCreate<float> 50 50
ERS
cgravill commented 7 years ago

@xperiandri it looks very likely to be related

Note the issue is still present on Mono 5.2.0.224, macOS 10.12.6

xperiandri commented 6 years ago

So what to do? It is a blocking issue

cgravill commented 6 years ago

I've done the preliminary isolation and made IfSharp clearer about the failure. If someone with experience of debugging Mono itself were able to assist I suspect it would be quickly resolved. Know anyone who could help? Equally if someone could reproduce it in the command-line it could be raised directly there.

xperiandri commented 6 years ago

Try this https://notebooks.azure.com/n/k0AJC6eoc6k/notebooks/IntroToNeuralNetworks.ipynb Just create a matrix of 100x9 Cell 14, line let m = 100

open System

let m = 100
let n = 9
let rand = Random()
let w0 = rand.NextDouble()
let w1 = rand.NextDouble()
let gam = 0.01;

let ERS = Array2D.zeroCreate<float> m n
ERS
cgravill commented 6 years ago

What we really need is to reproduce it on the command-line, or figure out a way to avoid it. I tried doing that above with the compiler services but I'm not able to cause the issue so far.

There's presumably something specific about the IfSharp host + mono that is causing this issue.

cgravill commented 6 years ago

I've done a bit more narrowing:

let n = 100 // 92 and below is fine, 93 and above can cause issues
let float_array = Array.init n (fun _ -> Array.init n (fun _ -> 0.0))

then test cases:

float_array |> Seq.take 11 |> Seq.toArray //Taking 11 items is fine
float_array |> Seq.take 12 |> Seq.toArray //Taking 12 items causes exception, if the overall array is large enough

Expression evaluation failed: Object reference not set to an instance of an object NullReferenceExceptionObject reference not set to an instance of an object at Microsoft.FSharp.Core.Operators.FailurePattern (System.Exception error) [0x00001] in <59964427904cf4daa745038327449659>:0 at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.ErrorR (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn) [0x0001c] in <5a01e92addab8ea7a74503832ae9015a>:0 at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.ErrorRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000d8] in <5a01e92addab8ea7a74503832ae9015a>:0 

float_array |> Seq.skip 1 |> Seq.take 11 |> Seq.toArray //Skipping 1 then taking 11 items is fine
float_array |> Seq.take 12 //Taking 12 but not turning it into array is fine
float_array |> Seq.take 12 |> Seq.toArray |> Seq.take 12 //Taking 12, turning it into an array, then taking them is also OK

So that removes the randomness, and string operations from above. It's also all the same value 0.0 in each location.

Mono JIT compiler version 5.4.1.6 FSharp.Compiler.Service (17.0.1) (tried upgrading this too)

cgravill commented 6 years ago

I've also been able to reproduce this using Mono on the Windows Subsystem for Linux with Ubuntu 16.04.

Mono JIT compiler version 5.4.1.6 (tarball Wed Nov  8 20:37:08 UTC 2017)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        LLVM:          supported, not enabled.
        GC:            sgen (concurrent by default)

On the same machine using the Windows install of Mono

Mono JIT compiler version 5.4.1 (Visual Studio built mono)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           normal
        SIGSEGV:       normal
        Notification:  Thread + polling
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        LLVM:          supported, not enabled.
        GC:            sgen

it works without issue on the above test cases.

dgrechka commented 6 years ago

@dsyme , I've created a fsharpc repro for the issue.

System

Ubuntu 18.04 Mono JIT compiler version 5.12.0.226

Repro steps

  1. Download and unzip the repro archive
  2. Run run_repro.sh from it's directory. (it compiles 3 files with fsharpc, then executes compiled exe to trigger the issue)

Comments

The output that I get is in repro_run_output.txt file inside the archive.

The issue is triggered by executing @cgravill code from above (taking 12 elements out of zero filled jagged array)

The exception thrown is not NullReference Exception like mentioned above, but StackOverflowException. Despite the exception is different, the issue is the same as I got this StackOverflow exception by execution @cgravill repro code via Jupyter notebook. Issue behavior follows the comments by @cgravill .

cgravill commented 6 years ago

Thanks @dgrechka that substantially narrows down the issue!

I'm able to reproduce on the Linux System System for Linux (https://www.mono-project.com/download/stable/#download-lin):

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
sudo apt install apt-transport-https
echo "deb https://download.mono-project.com/repo/ubuntu stable-xenial main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list
sudo apt update

sudo apt install mono-devel

sudo apt-get install fsharp

and results in:

./run_repro.sh
System info
Linux WINDOWS-XXXXXXX 4.4.0-17134-Microsoft #81-Microsoft Sun May 20 01:14:00 PST 2018 x86_64 x86_64 x86_64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:        16.04
Codename:       xenial
Mono JIT compiler version 5.12.0.226 (tarball Thu May  3 09:48:32 UTC 2018)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        Interpreter:   yes
        LLVM:          supported, not enabled.
        GC:            sgen (concurrent by default)
Clearing outputs
Building executable with fsharpc...
Microsoft (R) F# Compiler version 4.1
Copyright (c) Microsoft Corporation. All Rights Reserved.
Gathering dependencies...
Triggering issue by running executable...

Unhandled Exception:
StackOverflowException
[ERROR] FATAL UNHANDLED EXCEPTION: System.StackOverflowException: The requested operation caused a stack overflow.

(full log attached) repo_WSL.txt

cgravill commented 6 years ago

The current direct dependencies are:

FSharp.Compiler.Service (13.0)
FSharp.Compiler.Tools (4.1.29)

I upgraded each to their latest:

nuget FSharp.Compiler.Service ~> 22.0.3
nuget FSharp.Compiler.Tools ~> 10.0.2

Some details are changed but the overall result is the same:

[ERROR] FATAL UNHANDLED EXCEPTION: System.StackOverflowException: The requested operation caused a stack overflow.
  at Microsoft.FSharp.Compiler.Layout.addL@388-4[b,a,c] (Microsoft.FSharp.Compiler.Layout+LayoutRenderer`2[a,b] rr, a z, System.Int32 pos, System.Int32 i, Internal.Utilities.StructuredFormat.Layout layout, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] k) <0x7fb7f150 + 0x00007> in <5abc127aa80f33aea74503837a12bc5a>:0
  at Microsoft.FSharp.Compiler.Layout+addL@388-5T[a,b,c].Invoke (a z, System.Int32 pos, System.Int32 i, Internal.Utilities.StructuredFormat.Layout layout, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] k) [0x00007] in <5abc127aa80f33aea74503837a12bc5a>:0
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@3307-3[T2,T3,T4,T5,TResult,T1].Invoke (T2 u, T3 v, T4 w, T5 x) [0x00001] in <5893d081904cf4daa745038381d09358>:0
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@3301-2[T2,T3,T4,TResult,T1].Invoke (T2 u, T3 v, T4 w) [0x00001] in <5893d081904cf4daa745038381d09358>:0
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@3266-1[T2,T3,TResult,T1].Invoke (T2 u, T3 v) [0x00001] in <5893d081904cf4daa745038381d09358>:0
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@3253[T2,TResult,T1].Invoke (T2 u) [0x00001] in <5893d081904cf4daa745038381d09358>:0
cgravill commented 6 years ago

Huge thanks to @dgrechka on further narrowing this issue. The workaround applied seems like a good tradeoff for now, and we can remove this later when the underlying issue is resolved in Mono.

xperiandri commented 6 years ago

When will this come to Azure Notebook?

cgravill commented 6 years ago

@xperiandri see this https://github.com/Microsoft/AzureNotebooks/issues/395#issuecomment-397894327 for the usual schedule on Azure Notebooks deployments.