[browser][publish] Windows has problems with AOT build/publish for long paths

[ilonatommy]

Reproduction: 1) WBT Wasm.Build.Tests.Blazor.BuildPublishTests - edit id to be $"blz_aot_{config}_{GetRandomId()}_TEST_OF_EXTREMELY_LONG_PATH", or anything that will result in path longer than 260 chars. 2) Run, first error will be connected with g_fopen code:

Unable to open trimming-eligible-methods-outfile specified file C:\Users\username\source\repos\runtime-fork\artifacts\bin\Wasm.Build.Tests\Release\net9.0\win-x64\wbt artifacts\blz_aot_Debug_tumnkn5b_5fc_TEST_OF_LONG_PATH\obj\Debug\net9.0\wasm\for-publish\tokens\Microsoft_Extensions_DependencyInjection_dll_compiled_methods.txt

and can be fixed by editing g_fopen (const gchar *path, const gchar *mode) from gfile.c to:

gchar *path_mod;
#ifdef HOST_WIN32
    // add long-path prefix
    path_mod = g_malloc(strlen(path) + 5);
    strcpy(path_mod, "\\\\?\\");
    strcat(path_mod, path);
    if (is_ascii_string (path_mod) && is_ascii_string (mode)) {
        fp = fopen (path_mod, mode);
    } else {
        gunichar2 *wPath = g_utf8_to_utf16 (path_mod, -1, 0, 0, 0);
        gunichar2 *wMode = g_utf8_to_utf16 (mode, -1, 0, 0, 0);

        if (!wPath || !wMode)
            return NULL;

        fp = _wfopen ((wchar_t *) wPath, (wchar_t *) wMode);
        g_free (wPath);
        g_free (wMode);
    }
        g_free (path_mod);
#else

Further errors come from aot_printerrf (acfg, "Failed to load methodspec 0x%x due to %s.\n", token, mono_error_get_message (error)); in aot-compiler.c.

mono_get_method_checked Failed to load method 0x60001cc from 'C:\Users\source\repos\runtime-fork\artifacts\bin\Wasm.Build.Tests\Debug\net9.0\win-x64\wbt artifacts\blz_aot_Release_1t1y0tcq_4j1_TEST_OF_EXTREMELY_LONG_PATH\obj\Release\net9.0\wasm\for-publish\aot-in\Microsoft.AspNetCore.Components.WebAssembly.dll'
          [] C:\Users\source\repos\runtime-fork\artifacts\bin\dotnet-latest\packs\Microsoft.NET.Runtime.WebAssembly.Sdk\9.0.0-dev\Sdk\WasmApp.Common.targets(697,5): error : due to Could not load file or assembly 'Microsoft.Extensions.Configuration.Abstractions, Version=9.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60' or one of its dependencies..
          [] C:\Users\source\repos\runtime-fork\artifacts\bin\dotnet-latest\packs\Microsoft.NET.Runtime.WebAssembly.Sdk\9.0.0-dev\Sdk\WasmApp.Common.targets(697,5): error : Run with MONO_LOG_LEVEL=debug for more information.
          [] C:\Users\source\repos\runtime-fork\artifacts\bin\dotnet-latest\packs\Microsoft.NET.Runtime.WebAssembly.Sdk\9.0.0-dev\Sdk\WasmApp.Common.targets(697,5): error : AOT of image C:\Users\source\repos\runtime-fork\artifacts\bin\Wasm.Build.Tests\Debug\net9.0\win-x64\wbt artifacts\blz_aot_Release_1t1y0tcq_4j1_TEST_OF_EXTREMELY_LONG_PATH\obj\Release\net9.0\wasm\for-publish\aot-in\Microsoft.AspNetCore.Components.WebAssembly.dll failed

They are caused by mono_get_method_checked from loader.c getting empty result but I am not able to track down the reason for it. cc @kg, @BrzVlad any ideas what might be going wrong there?

For debugging it is helpful to switch off parallelization of library precompiling in MonoAOTCompiler (PrecompileLibraryParallel).

[dotnet-policy-service[bot]]

Tagging subscribers to 'arch-wasm': @lewing See info in area-owners.md if you want to be subscribed.

[BrzVlad]

From what I know we open metadata stuff with mono_file_map_open. You could try applying your fix there. Also replacing any uses in the runtime of fopen to g_fopen wouldn't hurt.

[ilonatommy]

@BrzVlad thanks for ideas but the proposed fixes are not enough. Do we have a way to dump stack trace? We are getting to mono_error_set_simple_file_not_found in exception.c and setting the error there. I have hard time collecting the stack manually, are there any other ways than mono_wasm_print_stack_trace?

#ifdef HOST_WASM
#include <emscripten.h>
EM_ASM(
    var err = new Error();
    console.log ("Stacktrace: \n");
    console.log (err.stack);
    );
#endif

It does not print anything (debug mode).

[BrzVlad]

I don't think we have something to dump native stack. I used in the past for various reasons the backtrace api but I have no idea if it works on wasm/windows. You can check a potentially bitrotten implementation in mono_backtrace.

However, I thought the error happens during app compilation ? My understanding that this is during aot compilation to llvm bytecode ? So I don't think it makes sense to use browser api ?

[BrzVlad]

If you are running a desktop program, you could always make it assert when that error is set and then I think it would ask you automatically to attach with VS where you can get stack trace and other things.

[ilonatommy]

Yes, it's app build step and yes, JS code does not make sense, though that was the only stack trace dump I found in the code.

If you are running a desktop program

The problems are with WASM app.

[kg]

I would strongly advise against gating the long path (the \\?\ stuff) based on how long the input path is. Long paths have different parsing rules, so having our parsing behavior change arbitrarily based on how long the input path is will cause lots of really confusing failure modes for end users. We should either always use it (on targets where it's available - all modern windows, i think) or never use it.

[ilonatommy]

We should either always use it (on targets where it's available - all modern windows, i think) or never use it.

Thank you. I changed the snipped in the description to always append the prefix. Does any reason for Could not load file or assembly ... or one of its dependencies. come to your mind?

[kg]

Have you tried MONO_LOG_LEVEL=debug? The answer is probably in the debug log messages.

[kg]

From grepping around, you may need to edit mono_file_map_open to use long paths.

[lewing]

@ilonatommy is there work left for 9.0 here?

[ilonatommy]

Yes, I cannot find the proper place that has to be fixed and I got stuck with the PR that is linked to it (https://github.com/dotnet/runtime/pull/103766), with no idea how to proceed.

[pavelsavara]

@jeromelaban shared this workaround

<Target Name="_WorkaroundEmscriptenPathLength" Condition=" '$(OS)' == 'Windows_NT' " BeforeTargets="_SetupEmscripten">
    <Exec Command="mklink /J &quot;$(TMP)\emsdk&quot; &quot;$(EmscriptenSdkToolsPath)..\tools\&quot;" ContinueOnError="true" />

    <PropertyGroup>
        <EmscriptenSdkToolsPath>$(TMP)\emsdk\</EmscriptenSdkToolsPath>
        <EmscriptenUpstreamBinPath>$(EmscriptenSdkToolsPath)bin\</EmscriptenUpstreamBinPath>
        <EmscriptenUpstreamEmscriptenPath>$(EmscriptenSdkToolsPath)emscripten\</EmscriptenUpstreamEmscriptenPath>
    </PropertyGroup>
    <ItemGroup>
        <EmscriptenPrependPATH Remove="@(EmscriptenPrependPATH)" />
        <EmscriptenPrependPATH Include="$(EmscriptenUpstreamBinPath)" />
        <EmscriptenPrependPATH Include="$(EmscriptenUpstreamEmscriptenPath)" />
    </ItemGroup>
</Target>