Support F# Kernel (dotnet-interactive) on Wasm

[pawel-stadnicki-gertrud]

Edited: I describe the wanted feature for F# Kernel in dotnet interactive but the message of not supported operation seems to come from the standard F# scripting tools, hence I asked it first here

Is your feature request related to a problem? Please describe.

Dotnet Interactive is an amazing tool that is used within Polyglot Notebook extension for VS Code.

It is still pretty versatile and extendable so I have played with it from the early beginning and was able to write interesting kernel extensions.

But I always wanted to have the opportunity to write sth that is built around it from top to bottom, including a custom code editor that executes a lot of my F# DSL code behind the scenes.

I was able to create a PoC that works with Blazor but only on a server mode. It has a lot of limitations and burdens like supporting multiple users in an isolated and performant manner.

Describe the solution you'd like

I would like to move the custom code editor that runs dotnet-interactive F# kernel to the client with webassembly. Previously (<= 8.0.0) the same code that runs in a server mode was hanging in a webassembly mode:

"Requested value for x" message was never displayed, there were no single error in the console as well

I got a brief suggestion from dotnet-interactive team that it maybe a result of lacking mutlithreading support with webassembly and I gave up for a while.

At the beginning of this year I noticed that a real multithreading for Blazor is planned to go with .NET 9(https://github.com/dotnet/aspnetcore/issues/17730)

Despite the fact it has just been marked as tentative due to some several discovered challenges, there is an ongoing work on it.

And if fact with .NET 9 preview-1 I noticed that the code no longer hangs but just fails and with recent .NET 9 preview-2 the exact error is much more clear.

The RequestValue command produces CommandFailed event with the message:

System.PlatformNotSupportedException: Operation is not supported on this platform.

The native code that it complains is below, not sure if it provide any correct direction thought

Do you have any first impressions about the problematic code ? Is it something that is possible to resolve ? Does it display the real cause or the message is misleading ?

@KevinRansom @vzarytovskii @dsyme @T-Gro @colombod @jonsequitur

Also tried the same with CSharpKernel but it fails early on kernel creation with no message so F# goes much further here.

Let's assume this problem is manageable, I'm obviously not sure what will happen further, as the work on multithreading support is on an early stage.

I believe that using Nuget packages maybe an issue in WebAssembly (although I do not necessarily require it in my case) Can you spot other possible endeavours ?

Describe alternatives you've considered

none

[vzarytovskii]

I think it should be moved to dotnet/interactive or wasm repository with potential follow up here. We don't do anything special for wasm as a target, we just emit normal IL which is then getting compiled to wasm.

[pawel-stadnicki-gertrud]

I think it should be moved to dotnet/interactive or wasm repository with potential follow up here. We don't do anything special for wasm as a target, we just emit normal IL which is then getting compiled to wasm.

I briefly looked that F# Kernel just uses standard F# stuff, and the script helpers rely on FSharp.Compiler.Interactive.Shell and FsiEvaluationSession, this is why I asked this first here, but we can move this elsewhere.

[vzarytovskii]

I think it should be moved to dotnet/interactive or wasm repository with potential follow up here. We don't do anything special for wasm as a target, we just emit normal IL which is then getting compiled to wasm.

I briefly looked that F# Kernel just uses standard F# stuff, and the script helpers rely on FSharp.Compiler.Interactive.Shell and FsiEvaluationSession, this is why I asked this first here, but we can move this elsewhere.

Regarding F# async - WASM currently is not playing with it (see https://github.com/dotnet/fsharp/issues/15272 as an example), however I'm not sure if there's something we can improve or is it on wasm side.

Regarding the stack trace where it's failing - it looks like it's failing somewhere in the glue code?

I, unfortunately, not as knowledgeable in WASM and its internals as folks from the team, so I'm hoping maybe @maraf or @lambdageek might be able to help us figuring out what can be done by F# team and contributors to make it work.

[T-Gro]

The call stack points to this line: https://github.com/dotnet/interactive/blob/3ff59f4fbb7af49dd3d71aeb2db16f65f9f91c97/src/Microsoft.DotNet.Interactive.FSharp/FSharpScriptHelpers.fs#L46 (it is in the interactive repo, but the same file exists in test utilities in this repo)

With stdin define like this in FSharp.Core:

        [<CompiledName("ConsoleIn")>]
        let stdin<'T> = Console.In

If someone would know why that fails in wasm, this can be detected - maybe the interactive FSharpScriptHelpers could pass in a different TextReader instance.

[PawelStadnicki]

Thanks all for the initial look around.

It is all .NET 9 Preview-2 with sparse info on how to use it, and there is a huge chance of breaking changes or even not going out the final .NET 9. But it may be worth catching problems as early as possible.

It does not have to be an issue with the tooling/libraries, it can be just my wrong Blazor setup or it is not ready because of the Preview-2.

I published the minimum project that reproduces it so someone can confirm it happens for others: https://github.com/PawelStadnicki/FSharp-Kernel-Wasm-Test

[PawelStadnicki]

More info here by @majocha with a suggestion it is a wasm runtime issue https://github.com/PawelStadnicki/FSharp-Kernel-Wasm-Test/issues/1#issue-2193651695

Also https://github.com/dotnet/aspnetcore/issues/54317#issuecomment-1979301983:

There are going to be lots of things broken with multithreading right now so it's not necessary to file issues for them individually, at least until we get to the point where we largely think things should work. But thanks for filing it :)

So we have to wait a few more previews and retry

[maraf]

I'm sorry for late reply.

IIRC the issue correctly, it's about using Console.In or stdin, that is not supported on WebAssembly https://github.com/maraf/runtime/blob/main/src/libraries/System.Console/src/System/ConsolePal.WebAssembly.cs#L108

There isn't such a thing as stdin in the browser. You can use JavaScript interop to glue a way to ask user for input.

[T-Gro]

@PawelStadnicki Pawel, I do not have full understanding of how you are linking things together.

But do you have an option of changing the default input reader like this https://learn.microsoft.com/en-us/dotnet/api/system.console.setin?view=net-8.0 ? Your own TextReader can be passed in, perharps based on your own input control.

[maraf]

@PawelStadnicki ☝️ (fixing the ping above 😉)

[PawelStadnicki]

Thanks @maraf and @T-Gro

I'm just sending some code to the F# kernel and asking for the result (string). Communication with the F # kernel involves commands and events, including diagnostics, so at the moment, I have no idea if my glue code is needed to satisfy in/out console as it is just throwing a platform not supported exception.

When that is changed maybe a dummy TextReader is enough? I'm not sure for what and if at all inReader/outWriter are used FsiEvaluationSession while serving Dotnet-Interactive, probably only if configured to do so (there is plenty of "if any" or "if needed" in the F# repo but I have only checked briefly).

So, do I understand correctly that I should provide a custom text reader somewhere in a Blazor Program, as Tomas suggested after the Wasm team changed this:

internal static TextReader GetOrCreateReader() => throw new PlatformNotSupportedException();?

Marek, so the next try will be with Preview-3?

[maraf]

Console.SetIn(new StringReader("Hello from custom stdin!"));
var line = Console.ReadLine();

This works on wasm. So if F# kernel somewhere touches stdin reader, this seems like a workaround

EDIT: It seems F# is touching other things

System.PlatformNotSupportedException: System.Diagnostics.Process is not supported on this platform. 
  at System.Diagnostics.Process.GetCurrentProcess() 
  at FSharp.Compiler.Interactive.Shell.FsiTimeReporter..ctor(TextWriter outWriter) in D:\a\_work\1\s\src\Compiler\Interactive\fsi.fs:line 267 
  at FSharp.Compiler.Interactive.Shell.FsiEvaluationSession..ctor(FsiEvaluationSessionHostConfig fsi, String[] argv, TextReader inReader, TextWriter outWriter, TextWriter errorWriter, Boolean fsiCollectible, FSharpOption`1 legacyReferenceResolver) in D:\a\_work\1\s\src\Compiler\Interactive\fsi.fs:line 4482 

[PawelStadnicki]

Console.SetIn(new StringReader("Hello from custom stdin!"));
var line = Console.ReadLine();

This works on wasm. So if F# kernel somewhere touches stdin reader, this seems like a workaround

EDIT: It seems F# is touching other things

System.PlatformNotSupportedException: System.Diagnostics.Process is not supported on this platform. 
  at System.Diagnostics.Process.GetCurrentProcess() 
  at FSharp.Compiler.Interactive.Shell.FsiTimeReporter..ctor(TextWriter outWriter) in D:\a\_work\1\s\src\Compiler\Interactive\fsi.fs:line 267 
  at FSharp.Compiler.Interactive.Shell.FsiEvaluationSession..ctor(FsiEvaluationSessionHostConfig fsi, String[] argv, TextReader inReader, TextWriter outWriter, TextWriter errorWriter, Boolean fsiCollectible, FSharpOption`1 legacyReferenceResolver) in D:\a\_work\1\s\src\Compiler\Interactive\fsi.fs:line 4482 

correct, not sure if this timing support can be avoided if there is no diagnostic logger provided @T-Gro ?

It is only used there and in DynamicCompiler where there is already some indication for an flag being moved to FsiOptions:

[T-Gro]

This could certainly work. Also, the retrieval of Process info could be made lazy to avoid this particular occurrence of the exception.

However, there might be a lot of "next issue behind the corner" situations, not making the fixing process very efficient.

@maraf : Is it possible to scan e.g. a .dll for APIs not supported by WASI at the moment? It would make it a lot faster to spot potential issues.

[PawelStadnicki]

However, there might be a lot of "next issue behind the corner" situations, not making the fixing process very efficient

Specifically, this is only the second line from ~250 lines of setup for FsiEvaluationSession (not counting the methods) and the very next one is Directory.GetCurrentDirectory() ...

Probably a hard decision to adjust all of it or make a separate, more platform-agnostic version, requiring broader planning

[maraf]

All the APIs are marked with SupportedOSPlatform https://learn.microsoft.com/en-us/dotnet/standard/analyzers/platform-compat-analyzer. So we need to analyze F# kernel for browser-wasm RID

[PawelStadnicki]

@T-Gro despite all the necessary work to support fsi on warm, I wonder if it is possible at all. Let's say we want to load Nuget package, is some special force of fsc.exe needed there? I'm not expert here and did not have time to dive into the details yet but I'm willing to work on it if this all is possible at all.

[maraf]

Let's say we want to load Nuget package

The AssemblyLoadContext works on wasm (so you can dynamically load additional assemblies)

[vzarytovskii]

@T-Gro despite all the necessary work to support fsi on warm, I wonder if it is possible at all. Let's say we want to load Nuget package, is some special force of fsc.exe needed there? I'm not expert here and did not have time to dive into the details yet but I'm willing to work on it if this all is possible at all.

If you mean #r nuget: ..., then it's likely not going to work, since dependency manager will literally create a fake fsproj and run some nuget commands on it. Loading arbitrary dll should not be a problem on the other hand.