Failed to create CoreCLR, HRESULT: 0x8007FF02

[arrowd]

The first roadblock I hit when trying to port this:

/wrkdirs/usr/ports/lang/dotnet/work/dotnet nuget add source 'https://fbsdnugetfeed.mooo.com/v3/index.json' --name ghsec --configfile runtime/NuGet.config
Failed to create CoreCLR, HRESULT: 0x8007FF02

This is a jail context so it might have something to do with that. Any ideas @sec @Thefrank ?

[arrowd]

Yes, outside of jail this command at least starts doing something:

/usr/local/poudriere/ports/default/lang/dotnet/work/dotnet nuget add source 'https://fbsdnugetfeed.mooo.com/v3/index.json' --name ghsec --configfile runtime/NuGet.config

Welcome to .NET 8.0!
---------------------
SDK Version: 8.0.100

----------------
Installed an ASP.NET Core HTTPS development certificate.
...

[arrowd]

Bakctrace that might be useful:

(gdb) bt
#0  _write () at _write.S:4
#1  0x0000000801345157 in _swrite (fp=0x8014064e0, buf=buf@entry=0x801b12a90 "Failed to create CoreCLR, HRESULT: 0x8007FF02", n=45) at /usr/src/lib/libc/stdio/stdio.c:117
#2  0x000000080134047c in __sfvwrite (fp=fp@entry=0x8014064e0, uio=uio@entry=0x7fffffffe0a0) at /usr/src/lib/libc/stdio/fvwrite.c:89
#3  0x000000080133eced in fputs_unlocked (s=0x801b12a90 "Failed to create CoreCLR, HRESULT: 0x8007FF02", fp=0x8014064e0) at /usr/src/lib/libc/stdio/fputs.c:63
#4  fputs (s=0x801b12a90 "Failed to create CoreCLR, HRESULT: 0x8007FF02", fp=0x8014064e0) at /usr/src/lib/libc/stdio/fputs.c:75
#5  0x00000008016b5970 in pal::err_fputs (message=0x801b12a90 "Failed to create CoreCLR, HRESULT: 0x8007FF02") at /usr/home/sec/dotnet-dotnet/src/runtime/artifacts/source-build/self/src/src/native/corehost/hostmisc/pal.h:227
#6  trace::error (format=0x8016831c8 "Failed to create CoreCLR, HRESULT: 0x%X") at /usr/home/sec/dotnet-dotnet/src/runtime/artifacts/source-build/self/src/src/native/corehost/hostmisc/trace.cpp:182
#7  0x00000008016af08a in (anonymous namespace)::create_coreclr () at /usr/home/sec/dotnet-dotnet/src/runtime/artifacts/source-build/self/src/src/native/corehost/hostpolicy/hostpolicy.cpp:84
#8  0x00000008016ae918 in corehost_main (argc=10, argv=0x801a2f050) at /usr/home/sec/dotnet-dotnet/src/runtime/artifacts/source-build/self/src/src/native/corehost/hostpolicy/hostpolicy.cpp:422
#9  0x000000080164fcb3 in execute_app (impl_dll_dir="/wrkdirs/usr/ports/lang/dotnet/work/shared/Microsoft.NETCore.App/8.0.0", argc=10, argv=0x801a2f050, init=<optimized out>)
    at /usr/home/sec/dotnet-dotnet/src/runtime/artifacts/source-build/self/src/src/native/corehost/fxr/fx_muxer.cpp:145
#10 (anonymous namespace)::read_config_and_execute (host_command=..., host_info=..., app_candidate="/wrkdirs/usr/ports/lang/dotnet/work/sdk/8.0.100/dotnet.dll", opts=..., new_argc=10, new_argv=0x801a2f050, mode=<optimized out>, is_sdk_command=<optimized out>, 
    out_buffer=<optimized out>, buffer_size=<optimized out>, required_buffer_size=<optimized out>) at /usr/home/sec/dotnet-dotnet/src/runtime/artifacts/source-build/self/src/src/native/corehost/fxr/fx_muxer.cpp:532
#11 fx_muxer_t::handle_exec_host_command (host_command="", host_info=..., app_candidate="/wrkdirs/usr/ports/lang/dotnet/work/sdk/8.0.100/dotnet.dll", opts=std::unordered_map with 0 elements, argc=<optimized out>, argv=<optimized out>, argoff=1, mode=muxer, 
    is_sdk_command=<optimized out>, result_buffer=0x0, buffer_size=0, required_buffer_size=0x0) at /usr/home/sec/dotnet-dotnet/src/runtime/artifacts/source-build/self/src/src/native/corehost/fxr/fx_muxer.cpp:1007
#12 0x000000080164f362 in fx_muxer_t::handle_cli (host_info=..., argc=argc@entry=9, argv=argv@entry=0x7fffffffea30, app_candidate="nuget") at /usr/home/sec/dotnet-dotnet/src/runtime/artifacts/source-build/self/src/src/native/corehost/fxr/fx_muxer.cpp:1093
#13 0x000000080164ec7c in fx_muxer_t::execute (host_command="", argc=9, argv=0x7fffffffea30, host_info=..., result_buffer=0x0, buffer_size=0, required_buffer_size=0x0)
    at /usr/home/sec/dotnet-dotnet/src/runtime/artifacts/source-build/self/src/src/native/corehost/fxr/fx_muxer.cpp:567
#14 0x000000080164b6ad in hostfxr_main_startupinfo (argc=9, argv=0x7fffffffea30, host_path=0x801a1b030 "/wrkdirs/usr/ports/lang/dotnet/work/dotnet", dotnet_root=0x801a1b060 "/wrkdirs/usr/ports/lang/dotnet/work/", 
    app_path=0x801a1b090 "/wrkdirs/usr/ports/lang/dotnet/work/dotnet.dll") at /usr/home/sec/dotnet-dotnet/src/runtime/artifacts/source-build/self/src/src/native/corehost/fxr/hostfxr.cpp:62
#15 0x000000000102f2c7 in exe_start (argc=argc@entry=9, argv=argv@entry=0x7fffffffea30) at /usr/home/sec/dotnet-dotnet/src/runtime/artifacts/source-build/self/src/src/native/corehost/corehost.cpp:240
#16 0x000000000102f526 in main (argc=9, argv=0x7fffffffea30) at /usr/home/sec/dotnet-dotnet/src/runtime/artifacts/source-build/self/src/src/native/corehost/corehost.cpp:308

[sec]

Hi. Make sure that you have allow.mlock=true when working under jail. Also make sure all the packages needed by dotnet are accessible also under jail (icu libunwind krb5 openssl, etc.). Next thing is elfctl -e +noaslr dotnet to disable ASLR on dotnet binary.

Just fyi - what procedure are you trying to port? When doing build from VMR, you only need bootstrapped SDK and artifacts produced (plus extra patches) to perform the build. If you try to port build using my scripts/steps, there will be rough edges :) I would go with VMR route

[arrowd]

Yep, I just tracked this down to

mmap(0x0,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 52025821757440 (0x2f5134a00000)
mlock(0x2f5134a00000,4096)                       ERR#1 'Operation not permitted'

which is 99% the problem you're describing.

Just fyi - what procedure are you trying to port? When doing build from VMR, you only need bootstrapped SDK and artifacts produced (plus extra patches) to perform the build. If you try to port build using my scripts/steps, there will be rough edges :) I would go with VMR route

I'm doing porting based on your steps. From my understanding, following these steps would produce me both VMR and SDK suitable for further bootstrapping?

[sec]

Nope, my steps will produce only SDK and runtime nuget's that can be used to build/run application - those are hacky ways of getting the SDK as it require some manual patching etc.

The easiest way now, at least for v8.0 is to go straight with VMR build - https://github.com/Thefrank/freebsd-dotnet-sourcebuild/tree/main/patches/Net8.0.100 - this will produce all you need, SDK, artifacts, etc., that can also be used to re-build the same VMR :) You can use bootstrapped SDK and private artifacts either from my releases or from frank's - they are the same, only difference is that he did then on 12 and mine are build on 13 - I also took his runtime patch for GC memory thing, which you can also include in build steps until it's not up-streamed.

Doing build straight from VMR repository should be 3 line operation assuming everything is in place :)

[arrowd]

Ok, I'll try going that route.

As for the intial problem, it can be fixed by putting JAIL_PARAMS=allow.mlock=true into poudriere.conf.