SteveSandersonMS / DotNetIsolator

A library for running isolated .NET runtimes inside .NET
MIT License
604 stars 40 forks source link

Too many invocations cause WasmtimeException "sgen-stw.c:77, condition ... not met" #17

Open warappa opened 1 year ago

warappa commented 1 year ago

Setup

I wanted to try out the limits of DotNetIsolator and changed the ConsoleSample.Program.cs file to have call runtime.Invoke(...) many more times (because you don't know how often a 3rd party code would call Invoke(...)). I also emptied the lambda body to speed up the invocation and to not have parameters, captured variables or return values have any influence.

Any non-empty lambda body causes the error to happen at an even lower invocation count.

Issue

Up to 35317 invocations there is no problem: The final output of the host is printed in the console (with impressive 0,04ms/call!).

But when incrementing numCalls by just 1 to 35318, it breaks with a WasmtimeException (see below).

Maybe this is an issue of Wasmtime itself, but I'm not sure as I'm not experienced in this kind of magic 😉.

Edited ConsoleSample

using var host = new IsolatedRuntimeHost().WithBinDirectoryAssemblyLoader();
using var runtime = new IsolatedRuntime(host);

//int numCalls = 35317; // GOOD
int numCalls = 35318; // BAD
var sw = new Stopwatch();
sw.Start();
for (var i = 0; i < numCalls; i++)
{
    runtime.Invoke(() =>
    {
    });
}
sw.Stop();
Console.WriteLine($"Done in {sw.ElapsedMilliseconds:F0}ms ({(double)sw.ElapsedMilliseconds / numCalls:F4} ms/call)");

Raised Exception

[wasm_trace_logger] * Assertion at /home/runner/work/dotnet-wasi-sdk/dotnet-wasi-sdk/modules/runtime/src/mono/mono/metadata/sgen-stw.c:77, condition `info->client_info.stack_start >= info->client_info.info.stack_start_limit && info->client_info.stack_start < info->client_info.info.stack_end' not met

Unhandled exception. Wasmtime.WasmtimeException: error while executing at wasm backtrace:
    0: 0x40754b - <unknown>!<wasm function 7705>
    1: 0x405089 - <unknown>!<wasm function 7618>
    2: 0x407a16 - <unknown>!<wasm function 7714>
    3: 0xe2e68 - <unknown>!<wasm function 709>
    4: 0x2d2c58 - <unknown>!<wasm function 4879>
    5: 0x3f7937 - <unknown>!<wasm function 7321>
    6: 0x3f77e3 - <unknown>!<wasm function 7319>
    7: 0x3f7a2b - <unknown>!<wasm function 7323>
    8: 0x3f7ab6 - <unknown>!<wasm function 7324>
    9: 0x2b7fce - <unknown>!<wasm function 4637>
   10: 0x2b7d14 - <unknown>!<wasm function 4635>
   11: 0x30b42c - <unknown>!<wasm function 5606>
   12: 0x30ade8 - <unknown>!<wasm function 5605>
   13: 0x30ac21 - <unknown>!<wasm function 5604>
   14: 0x30ab95 - <unknown>!<wasm function 5603>
   15: 0x3003b6 - <unknown>!<wasm function 5444>
   16: 0x2bd5e5 - <unknown>!<wasm function 4725>
   17: 0x2292bf - <unknown>!<wasm function 3284>
   18: 0x229110 - <unknown>!<wasm function 3283>
   19: 0x92199 - <unknown>!<wasm function 333>
   20: 0x62d24 - <unknown>!<wasm function 332>
   21: 0x367621 - <unknown>!<wasm function 6309>
   22: 0x2110a7 - <unknown>!<wasm function 3085>
   23: 0x212d90 - <unknown>!<wasm function 3101>
   24: 0x21c3f8 - <unknown>!<wasm function 3187>
   25: 0xe3202 - <unknown>!<wasm function 714>
   26: 0xe3427 - <unknown>!<wasm function 715>
   27: 0x3cd2 - <unknown>!<wasm function 43>
   28: 0x4477 - <unknown>!<wasm function 46>
   29: 0x419242 - <unknown>!<wasm function 7924>

Caused by:
    Exited with i32 exit status 1
   at Wasmtime.Function.Invoke(Span`1 argumentsAndResults, StoreContext storeContext)
   at Wasmtime.Function.<>c__DisplayClass187_0`3.<WrapFunc>b__0(T1 p0, T2 p1)
   at DotNetIsolator.IsolatedRuntime.CopyObject[T](T value) in C:\source\DotNetIsolator\src\DotNetIsolator\IsolatedRuntime.cs:line 115
   at DotNetIsolator.IsolatedRuntime.Invoke(Action value) in C:\source\DotNetIsolator\src\DotNetIsolator\IsolatedRuntime.cs:line 308
   at Program.Main(String[] args) in C:\source\DotNetIsolator\sample\ConsoleSample\Program.cs:line 17

Expected Result

There should be no upper limit in invocation count.

Additional Information

Tested on 6dbe1a3b1890beea8a375d854ffe00784fa147dd This error also happened on previous versions of DotNetIsolator.

dotnet --info
.NET SDK: Version: 8.0.100-preview.3.23178.7 Commit: e300b0e1e6 Runtime Environment: OS Name: Windows OS Version: 10.0.22621 OS Platform: Windows RID: win10-x64 Base Path: C:\Program Files\dotnet\sdk\8.0.100-preview.3.23178.7\ .NET workloads installed: There are no installed workloads to display. Host: Version: 8.0.0-preview.3.23174.8 Architecture: x64 Commit: 47bad717bd .NET SDKs installed: 3.1.426 [C:\Program Files\dotnet\sdk] 5.0.408 [C:\Program Files\dotnet\sdk] 5.0.416 [C:\Program Files\dotnet\sdk] 6.0.311 [C:\Program Files\dotnet\sdk] 7.0.100-preview.7.22377.5 [C:\Program Files\dotnet\sdk] 7.0.100-rc.1.22431.12 [C:\Program Files\dotnet\sdk] 7.0.105 [C:\Program Files\dotnet\sdk] 7.0.302 [C:\Program Files\dotnet\sdk] 7.0.400-preview.23225.8 [C:\Program Files\dotnet\sdk] 8.0.100-preview.3.23178.7 [C:\Program Files\dotnet\sdk] .NET runtimes installed: Microsoft.AspNetCore.All 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All] Microsoft.AspNetCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 7.0.0-preview.7.22376.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 7.0.0-rc.1.22427.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 8.0.0-preview.3.23177.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.NETCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.21 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 7.0.0-preview.7.22375.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 7.0.0-rc.1.22426.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 8.0.0-preview.3.23174.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.WindowsDesktop.App 3.1.21 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 5.0.12 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 6.0.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 7.0.0-preview.7.22377.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 7.0.0-rc.1.22427.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 8.0.0-preview.3.23178.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Other architectures found: arm64 [C:\Program Files\dotnet] registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\arm64\InstallLocation] x86 [C:\Program Files (x86)\dotnet] registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation] Environment variables: Not set global.json file: Not found
dotnet workload list
Installed Workload Id Manifest Version Installation Source --------------------------------------------------------------------------------------------- macos 13.1.462-net8-p3/8.0.100-preview.3 VS 17.7.33711.374 maccatalyst 16.2.462-net8-p3/8.0.100-preview.3 VS 17.7.33711.374 maui-windows 8.0.0-preview.3.8149/8.0.100-preview.3 VS 17.7.33711.374 maui-maccatalyst 8.0.0-preview.3.8149/8.0.100-preview.3 VS 17.7.33711.374 maui-ios 8.0.0-preview.3.8149/8.0.100-preview.3 VS 17.7.33711.374 wasm-tools 8.0.0-preview.3.23174.8/8.0.100-preview.3 VS 17.7.33711.374 wasm-tools-net6 8.0.0-preview.3.23174.8/8.0.100-preview.3 VS 17.7.33711.374 runtimes-windows 8.0.0-preview.3.23174.8/8.0.100-preview.3 VS 17.7.33711.374 ios 16.2.462-net8-p3/8.0.100-preview.3 VS 17.7.33711.374 android 34.0.0-preview.3.224/8.0.100-preview.3 VS 17.7.33711.374 maui-android 8.0.0-preview.3.8149/8.0.100-preview.3 VS 17.7.33711.374

Personal Note

That's some awesome stuff your doing here! Please keep it on!

CalebSerafin commented 1 year ago

I've observed that this is not only caused by too many invocations, but also by too much data transfer. By limiting to only 1 invocation, using the following code I manually binary searched the maximum amount of bytes I could return.

I cause the same error by either: Incrementing that amount by 1; Or the sum of multiple invocations exceeds that number.

WasiConfiguration wasiConfiguration = new WasiConfiguration().WithInheritedStandardOutput().WithInheritedStandardError();
using var host = new IsolatedRuntimeHost().WithAssemblyLoader(LoadAssembly);
using var isolatedRuntime = new IsolatedRuntime(host);

// Maximum bytes without crash: 0b0111_0110_0100_1010_1110_0011 = 7752419 = 7.39 MiB
const int bytesToGenerate = 0b0111_0110_0100_1010_1110_0011;

IsolatedObject byteGenerator = isolatedRuntime.CreateObject<ReturnManyPseudoRandomByteGenerator>();
byteGenerator.Invoke<int, int, byte[]>("GenerateBytes", 1, bytesToGenerate);

class ReturnManyPseudoRandomByteGenerator {
    public byte[] GenerateBytes(int seed, int count) {
        Random random = new(seed);
        byte[] bytes = new byte[count];
        random.NextBytes(bytes);
        return bytes;
    }
}