Open sec opened 2 years ago
OSX/BSD builds use the target system's libunwind: from src/coreclr/pal/src/CMakeLists.txt
if(CLR_CMAKE_TARGET_OSX OR CLR_CMAKE_TARGET_FREEBSD)
# On OSX and *BSD, we use the libunwind that's part of the OS
set(CLR_CMAKE_USE_SYSTEM_LIBUNWIND 1)
endif(CLR_CMAKE_TARGET_OSX OR CLR_CMAKE_TARGET_FREEBSD)
which makes it add only include_directories(SYSTEM $ENV{ROOTFS_DIR}/usr/local/include)
Yup, I think we just need to update the list of compatible context registers for freebsd-arm64 to make it work.
If that would only make sense to me I would maybe help more, still learing on the fly. I found #40149 and doing quick change:
diff --git a/src/coreclr/pal/src/include/pal/context.h b/src/coreclr/pal/src/include/pal/context.h
index fec5139fd5e..2cc4941d653 100644
--- a/src/coreclr/pal/src/include/pal/context.h
+++ b/src/coreclr/pal/src/include/pal/context.h
@@ -407,7 +407,7 @@ inline void *FPREG_Xstate_Ymmh(const ucontext_t *uc)
#if defined(HOST_ARM64)
-#ifndef TARGET_OSX
+#if !defined(TARGET_OSX) && !defined(TARGET_FREEBSD)
#define MCREG_X0(mc) ((mc).regs[0])
#define MCREG_X1(mc) ((mc).regs[1])
@@ -482,7 +482,7 @@ const fpsimd_context* GetConstNativeSigSimdContext(const native_context_t *mc)
return GetNativeSigSimdContext(const_cast<native_context_t*>(mc));
}
-#else // TARGET_OSX
+#elif defined(TARGET_OSX) // TARGET_OSX
#define MCREG_X0(mc) ((mc)->__ss.__x[0])
#define MCREG_X1(mc) ((mc)->__ss.__x[1])
native compile fail with:
In file included from /usr/home/sec/runtime/src/coreclr/pal/src/exception/seh.cpp:407:
/usr/home/sec/runtime/src/coreclr/pal/src/exception/seh-unwind.cpp:177:2: error: unsupported architecture
#error unsupported architecture
^
/usr/home/sec/runtime/src/coreclr/pal/src/exception/seh-unwind.cpp:183:5: error: use of undeclared identifier 'ASSIGN_UNWIND_REGS'
ASSIGN_UNWIND_REGS
^
2 errors generated.
--- pal/src/CMakeFiles/coreclrpal.dir/exception/seh.cpp.o ---
*** [pal/src/CMakeFiles/coreclrpal.dir/exception/seh.cpp.o] Error code 1
Is this the context registers that needs to be updated (looked at the code and there's no ARM64 for any of the targets)?
@sec, what are the detected features (the HAVE_xxx) for the FreeBSD arm64 build? When you do a clean build, it lists them to the console. From those, I should be able to tell where to update the context registers extraction.
@janvorli There you go:
-- Performing Test COMPILER_SUPPORTS_F_STACK_PROTECTOR_STRONG - Success
-- Performing Test COMPILER_SUPPORTS_W_IMPLICIT_FALLTHROUGH - Success
-- Performing Test HAVE_LTO - Failed
-- Performing Test COMPILER_SUPPORTS_F_ALIGNED_NEW - Success
-- Performing Test HAVE_DIRENT_D_TYPE - Success
-- Performing Test C_SUPPORTS_WUNGUARDED_AVAILABILITY - Success
-- Performing Test HAVE_IN_PKTINFO - Failed
-- Performing Test HAVE_IP_MREQN - Success
-- Performing Test HAVE_STATFS_VFS - Failed
-- Performing Test HAVE_STATFS_MOUNT - Success
-- Performing Test HAVE_FLOCK64 - Failed
-- Performing Test HAVE_IFADDRS - Success
-- Performing Test HAVE_UTSNAME_DOMAINNAME - Failed
-- Performing Test HAVE_STAT_BIRTHTIME - Success
-- Performing Test HAVE_STAT_FLAGS - Success
-- Performing Test HAVE_STAT_TIMESPEC - Success
-- Performing Test HAVE_STAT_TIM - Success
-- Performing Test HAVE_STAT_NSEC - Success
-- Performing Test HAVE_DIRENT_NAME_LEN - Success
-- Performing Test HAVE_STATFS_FSTYPENAME - Success
-- Performing Test HAVE_STATVFS_FSTYPENAME - Failed
-- Performing Test HAVE_GNU_STRERROR_R - Failed
-- Performing Test HAVE_READDIR_R - Success
-- Performing Test KEVENT_HAS_VOID_UDATA - Success
-- Performing Test HAVE_FDS_BITS - Success
-- Performing Test HAVE_PRIVATE_FDS_BITS - Success
-- Performing Test HAVE_SENDFILE_4 - Failed
-- Performing Test HAVE_SENDFILE_6 - Failed
-- Performing Test HAVE_SENDFILE_7 - Success
-- Performing Test HAVE_GETNAMEINFO_SIGNED_FLAGS - Success
-- Performing Test HAVE_SHM_OPEN_THAT_WORKS_WELL_ENOUGH_WITH_MMAP - Success
-- Performing Test HAVE_CLOCK_MONOTONIC - Success
-- Performing Test HAVE_CLOCK_REALTIME - Success
-- Performing Test BIND_ADDRLEN_UNSIGNED - Success
-- Performing Test IPV6MR_INTERFACE_UNSIGNED - Success
-- Performing Test INOTIFY_RM_WATCH_WD_UNSIGNED - Failed
-- Performing Test HAVE_MKSTEMPS - Success
-- Performing Test HAVE_MKSTEMP - Success
-- Performing Test HAVE_NETINET_TCP_VAR_H - Success
-- Performing Test HAVE_NETINET_UDP_VAR_H - Success
-- Performing Test HAVE_NETINET_IP_VAR_H - Success
-- Performing Test HAVE_NETINET_ICMP_VAR_H - Success
-- Performing Test HAVE_TCP_H_TCPSTATE_ENUM - Failed
-- Performing Test HAVE_GETDOMAINNAME_SIZET - Failed
-- Performing Test HAVE_IN_EXCL_UNLINK - Success
-- Performing Test HAVE_TCP_H_TCP_KEEPALIVE - Failed
-- Performing Test HAVE_BUILTIN_MUL_OVERFLOW - Success
-- Performing Test HAVE_UDAT_STANDALONE_SHORTER_WEEKDAYS - Success
-- Performing Test HAVE_VM_FLAGS_SUPERPAGE_SIZE_ANY - Failed
-- Performing Test HAVE_MAP_HUGETLB - Failed
-- Performing Test HAVE_LTTNG_TRACEPOINT_H - Failed
-- Performing Test HAVE_CPUSET_T - Success
-- Performing Test HAVE_TM_GMTOFF - Success
-- Performing Test HAVE_GREGSET_T - Failed
-- Performing Test HAVE___GREGSET_T - Failed
-- Performing Test HAVE_FPSTATE_GLIBC_RESERVED1 - Failed
-- Performing Test HAVE_SYSINFO_WITH_MEM_UNIT - Failed
-- Performing Test HAVE_FPREGS_WITH_CW - Failed
-- Performing Test REALPATH_SUPPORTS_NONEXISTENT_FILES - Failed
-- Performing Test SSCANF_SUPPORT_ll - Success
-- Performing Test SSCANF_CANNOT_HANDLE_MISSING_EXPONENT - Failed
-- Performing Test HAVE_LARGE_SNPRINTF_SUPPORT - Success
-- Performing Test HAVE_BROKEN_FIFO_SELECT - Failed
-- Performing Test HAVE_BROKEN_FIFO_KEVENT - Failed
-- Performing Test HAVE_SCHED_GET_PRIORITY - Success
-- Performing Test HAVE_WORKING_GETTIMEOFDAY - Success
-- Performing Test HAVE_WORKING_CLOCK_GETTIME - Success
-- Performing Test HAVE_CLOCK_MONOTONIC_COARSE - Success
-- Performing Test HAVE_CLOCK_THREAD_CPUTIME - Success
-- Performing Test MMAP_ANON_IGNORES_PROTECTION - Failed
-- Performing Test ONE_SHARED_MAPPING_PER_FILEREGION_PER_PROCESS - Failed
-- Performing Test PTHREAD_CREATE_MODIFIES_ERRNO - Failed
-- Performing Test SEM_INIT_MODIFIES_ERRNO - Failed
-- Performing Test HAVE_PROCFS_CTL - Failed
-- Performing Test HAVE_PROCFS_MAPS - Failed
-- Performing Test HAVE_PROCFS_STAT - Failed
-- Performing Test HAVE_PROCFS_STATUS - Success
-- Performing Test HAVE_COMPATIBLE_ACOS - Success
-- Performing Test HAVE_COMPATIBLE_ASIN - Success
-- Performing Test HAVE_COMPATIBLE_POW - Success
-- Performing Test HAVE_VALID_NEGATIVE_INF_POW - Success
-- Performing Test HAVE_VALID_POSITIVE_INF_POW - Success
-- Performing Test HAVE_COMPATIBLE_ATAN2 - Success
-- Performing Test HAVE_COMPATIBLE_EXP - Failed
-- Performing Test HAVE_COMPATIBLE_ILOGB0 - Failed
-- Performing Test HAVE_COMPATIBLE_ILOGBNAN - Success
-- Performing Test HAVE_COMPATIBLE_LOG - Success
-- Performing Test HAVE_COMPATIBLE_LOG10 - Success
-- Performing Test UNGETC_NOT_RETURN_EOF - Failed
-- Performing Test HAS_POSIX_SEMAPHORES - Success
-- Performing Test GETPWUID_R_SETS_ERRNO - Failed
-- Performing Test FILE_OPS_CHECK_FERROR_OF_PREVIOUS_CALL - Failed
-- Performing Test UNWIND_CONTEXT_IS_UCONTEXT_T - Success
-- Performing Test HAVE_XSWDEV - Success
-- Performing Test HAVE_XSW_USAGE - Failed
-- Performing Test HAVE_PUBLIC_XSTATE_STRUCT - Failed
-- Performing Test HAVE_PR_SET_PTRACER - Failed
-- Performing Test HAVE_FULLY_FEATURED_PTHREAD_MUTEXES - Success
-- Performing Test HAVE_PTHREAD_THREADID_NP - Failed
-- Performing Test HAVE_PTHREAD_GETTHREADID_NP - Success
edit: also attached full log from build attempt. build_log.txt
It seems to me that your last change was correct. But you also need to add #elif (defined(HOST_UNIX) && defined(HOST_ARM64)) before the #else with #error that fires. In that #elif, you'd add:
ASSIGN_REG(Pc) \
ASSIGN_REG(Sp) \
ASSIGN_REG(Fp) \
ASSIGN_REG(Lr) \
ASSIGN_REG(X19) \
ASSIGN_REG(X20) \
ASSIGN_REG(X21) \
ASSIGN_REG(X22) \
ASSIGN_REG(X23) \
ASSIGN_REG(X24) \
ASSIGN_REG(X25) \
ASSIGN_REG(X26) \
ASSIGN_REG(X27) \
ASSIGN_REG(X28)
I hope I got it right.
@janvorli Tried that, it failed with bunch of errors for every register:
In file included from /usr/home/sec/runtime/src/coreclr/pal/src/exception/seh.cpp:407:
/usr/home/sec/runtime/src/coreclr/pal/src/exception/seh-unwind.cpp:190:2: error: use of undeclared identifier 'MCREG_X21'
ASSIGN_UNWIND_REGS
^
/usr/home/sec/runtime/src/coreclr/pal/src/exception/seh-unwind.cpp:178:2: note: expanded from macro 'ASSIGN_UNWIND_REGS'
ASSIGN_REG(X21) \
^
/usr/home/sec/runtime/src/coreclr/pal/src/exception/seh-unwind.cpp:189:25: note: expanded from macro 'ASSIGN_REG'
#define ASSIGN_REG(reg) MCREG_##reg(unwContext->uc_mcontext) = winContext->reg;
^
<scratch space>:95:1: note: expanded from here
MCREG_X21
^
Leaving this empty, will go on but then fail again with similar errors. If I'm looking at this correctly, I think it's missing some #def for HOST_ARM64 and TARGET_FREEBSD inside src/coreclr/pal/src/include/pal/context.h
with those registers?
And also there's again error: unknown type name 'fpsimd_context'
- which FreeBSD doesn't have, but if I add || defined(TARGET_FREEBSD)
next to TARGET_OSX those errors seems to go away (don't know if that will work in the end though, as I going blind in here. Also the undefined registers errors are there, so my bet is that fixing those first would be real fix).
I wodner, if there's libunwind used (which state FreeBSD is supported under x64 and arm64) - shouldn't this be used (if I assume this code here is about that).
As I saw @sdmaclea did the OSX part of changes - maybe some hints? :)
Just to make sure, this is native build under ARM64 FreeBSD host.
Ah, I haven't noticed that your change around line 490 was like this:
-#else // TARGET_OSX
+#elif defined(TARGET_OSX) // TARGET_OSX
That way, you have eliminated both definitions of the MCREG_xxx for arm64. What does the ucontext_t struct look like on FreeBSD arm64? We'll either need to define the MCREG_xxx for the FreeBSD separately or reuse the one for Linux / OSX if it happens to be the same.
Ah, found it in FreeBSD sources. So it is different and we will need to have another #elif TARGET_FREEBSD
there. It seems that the definitions should look like:
#define MCREG_X21(mc) (mc.mc_gpregs.gp_x[21])
#define MCREG_X22(mc) (mc.mc_gpregs.gp_x[22])
#define MCREG_X23(mc) (mc.mc_gpregs.gp_x[23])
#define MCREG_X24(mc) (mc.mc_gpregs.gp_x[24])
#define MCREG_X25(mc) (mc.mc_gpregs.gp_x[25])
#define MCREG_X26(mc) (mc.mc_gpregs.gp_x[26])
#define MCREG_X27(mc) (mc.mc_gpregs.gp_x[27])
#define MCREG_X28(mc) (mc.mc_gpregs.gp_x[28])
#define MCREG_Fp(mc) (mc.mc_gpregs.gp_fp)
#define MCREG_Lr(mc) (mc.mc_gpregs.gp_lr)
#define MCREG_Sp(mc) (mc.mc_gpregs.gp_sp)
#define MCREG_Pc(mc) (mc.mc_gpregs.gp_pc)
As for the floating point access we will also need an extra code here. Take the following with a grain of salt, it might not be exactly right or might need extra change at another place too:
inline
struct fpregs* GetNativeSigSimdContext(native_context_t *mc)
{
return &(mc->mc_fpregs);
}
inline
const struct fpregs* GetConstNativeSigSimdContext(const native_context_t *mc)
{
return GetNativeSigSimdContext(const_cast<native_context_t*>(mc));
}
@janvorli Thanks, I was looking at the same files and was about to post :)
Can't find Pc
and Fp
in ucontext
in structs btw.
Managed to get past some errors, but now a lot of:
[ 6%] Built target eventpipe_objs
Consolidate compiler generated dependencies of target mdruntimerw_wks_obj
/usr/home/sec/runtime/src/coreclr/pal/src/thread/context.cpp:339:9: error: use of undeclared identifier 'BSDREG_Cpsr'
ASSIGN_ALL_REGS
^
/usr/home/sec/runtime/src/coreclr/pal/src/thread/context.cpp:286:9: note: expanded from macro 'ASSIGN_ALL_REGS'
ASSIGN_CONTROL_REGS \
^
/usr/home/sec/runtime/src/coreclr/pal/src/thread/context.cpp:137:9: note: expanded from macro 'ASSIGN_CONTROL_REGS'
ASSIGN_REG(Cpsr) \
^
/usr/home/sec/runtime/src/coreclr/pal/src/thread/context.cpp:333:62: note: expanded from macro 'ASSIGN_REG'
#define ASSIGN_REG(reg) MCREG_##reg(registers.uc_mcontext) = BSDREG_##reg(ptrace_registers);
^
<scratch space>:248:1: note: expanded from here
BSDREG_Cpsr
^
[ 8%] Built target ceefgen_obj
/usr/home/sec/runtime/src/coreclr/pal/src/thread/context.cpp:339:9: error: use of undeclared identifier 'BSDREG_Fp'
/usr/home/sec/runtime/src/coreclr/pal/src/thread/context.cpp:286:9: note: expanded from macro 'ASSIGN_ALL_REGS'
ASSIGN_CONTROL_REGS \
^
/usr/home/sec/runtime/src/coreclr/pal/src/thread/context.cpp:138:9: note: expanded from macro 'ASSIGN_CONTROL_REGS'
ASSIGN_REG(Fp) \
^
...
I haven't saw any existing BSDREG_X
for defined (even for OSX)
I'm starting to think does this road make sense. Shouldn't this reuse existing ARM64 code and/or use machine specific libunwind for the rest?
We cannot reuse the code, because the structure of the ucontext_t
is different from all other platforms, using different names of data fields for the registers etc. That's why we are adding new ones.
Can you please share the log from the clean build after you've applied my previous fix? It is not clear to me how come we are compiling code using the BSDREG_xxx stuff when the HAVE_BSD_REGS_T
doesn't seem to be defined.
build_log.txt freebsd_arm64_patch.txt
@janvorli Here's build output and git diff
of everything I have tweaked :)
edit: accoring to CMakeOutput.txt, BSD_REG check passed:
Run Build Command(s):/usr/local/bin/gmake -f Makefile cmTC_c2fef/fast && /usr/local/bin/gmake -f CMakeFiles/cmTC_c2fef.dir/build.make CMakeFiles/cmTC_c2fef.dir/build gmake[1]: Entering directory '/usr/home/sec/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug/CMakeFiles/CMakeTmp' Building C object CMakeFiles/cmTC_c2fef.dir/BSD_REGS_T.c.o /usr/local/bin/clang14 -D_FILE_OFFSET_BITS=64 -I/usr/local/include -fPIE -MD -MT CMakeFiles/cmTC_c2fef.dir/BSD_REGS_T.c.o -MF CMakeFiles/cmTC_c2fef.dir/BSD_REGS_T.c.o.d -o CMakeFiles/cmTC_c2fef.dir/BSD_REGS _T.c.o -c /usr/home/sec/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug/CMakeFiles/CheckTypeSize/BSD_REGS_T.c Linking C executable cmTC_c2fef /usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_c2fef.dir/link.txt --verbose=1 /usr/local/bin/clang14 -fuse-ld=lld -pie -Wl,--build-id=sha1 CMakeFiles/cmTC_c2fef.dir/BSD_REGS_T.c.o -o cmTC_c2fef gmake[1]: Leaving directory '/usr/home/sec/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug/CMakeFiles/CMakeTmp' CMakeError.log CMakeOutput.log
edit 2: added CMakeFiles output, maybe it will help also
Ah, I've forgotten that the check_type_size("struct reg" BSD_REGS_T)
doesn't log anything. In that case, we need to add definitions for BSDREG_xxx stuff. The HAVE_BSD_REGS_T
branch in the source assumes it is either x64 or x86. So in context.h
, look for the #if HAVE_BSD_REGS_T
and then #ifdef HOST_64BIT
under that. Inside that #ifdef
, we need to put the current stuff under #ifdef HOST_AMD64
and add #elif defined(HOST_ARM64)
branch where we put the following:
#define BSDREG_X0(reg) BSD_REGS_STYLE(reg,X[0],x[0])
.
.
.
#define BSDREG_X29(reg) BSD_REGS_STYLE(reg,X[29],x[29])
#define BSDREG_Fp(reg) BSD_REGS_STYLE(reg,Fp,fp)
#define BSDREG_Sp(reg) BSD_REGS_STYLE(reg,Sp,sp)
#define BSDREG_Pc(reg) BSD_REGS_STYLE(reg,Pc,pc)
#define BSDREG_Lr(reg) BSD_REGS_STYLE(reg,Lr,lr)
#define BSDREG_Cpsr(reg) BSD_REGS_STYLE(reg,Spsr,spsr)
And then in src/coreclr/pal/src/configure.cmake
, find elseif(CLR_CMAKE_TARGET_FREEBSD)
. In that branch, the set(BSD_REGS_STYLE "((reg).r_##rr)")
has to be different for amd64 and arm64. For arm64, it needs to be set(BSD_REGS_STYLE "((reg).rr)")
, because the naming convention of the registers is different there. So it should be:
if (CLR_CMAKE_HOST_ARCH_AMD64)
set(BSD_REGS_STYLE "((reg).r_##rr)")
elseif(CLR_CMAKE_HOST_ARCH_ARM64)
set(BSD_REGS_STYLE "((reg).rr)")
else()
message(FATAL_ERROR "Unknown FreeBSD architecture")
endif()
Thanks, will try that - any idea what about missing MCREG_Pc and MCREG_Fp - can't see those inside ucontext struct (or the name is not close to that) ?
Ah, sorry, the Fp is X29, so it should be
#define MCREG_Fp(mc) (mc.mc_gpregs.gp_x[29])
And the MCREG_Pc - hmm, that's strange. I can see that there is gp_lr and gp_elr. I think that maybe the gp_elr is in fact the Pc.
Also related to the x29 above, for the most recent change I've suggested, this is wrong:
#define BSDREG_X29(reg) BSD_REGS_STYLE(reg,X[29],x[29])
#define BSDREG_Fp(reg) BSD_REGS_STYLE(reg,Fp,fp)
It should be just
#define BSDREG_Fp(reg) BSD_REGS_STYLE(reg,X[29],x[29])
With everything so far: freebsd_arm_patch.txt
/home/frank/runtime/src/coreclr/pal/src/thread/context.cpp:339:9: error: no member named 'pc' in 'reg'
ASSIGN_ALL_REGS
^~~~~~~~~~~~~~~
/home/frank/runtime/src/coreclr/pal/src/thread/context.cpp:286:9: note: expanded from macro 'ASSIGN_ALL_REGS'
ASSIGN_CONTROL_REGS \
^~~~~~~~~~~~~~~~~~~
/home/frank/runtime/src/coreclr/pal/src/thread/context.cpp:141:9: note: expanded from macro 'ASSIGN_CONTROL_REGS'
ASSIGN_REG(Pc)
^~~~~~~~~~~~~~
/home/frank/runtime/src/coreclr/pal/src/thread/context.cpp:333:62: note: expanded from macro 'ASSIGN_REG'
#define ASSIGN_REG(reg) MCREG_##reg(registers.uc_mcontext) = BSDREG_##reg(ptrace_registers);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<scratch space>:310:1: note: expanded from here
BSDREG_Pc
^
/home/frank/runtime/src/coreclr/pal/src/include/pal/context.h:1020:46: note: expanded from macro 'BSDREG_Pc'
#define BSDREG_Pc(reg) BSD_REGS_STYLE(reg,Pc,pc)
~~~~~~~~~~~~~~~~~~~~~~^~~
/home/frank/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug/pal/src/config.h:158:44: note: expanded from macro 'BSD_REGS_STYLE'
#define BSD_REGS_STYLE(reg, RR, rr) ((reg).rr)
~~~~~ ^
/home/frank/runtime/src/coreclr/pal/src/thread/context.cpp:491:13: error: no member named 'pc' in 'reg'
ASSIGN_CONTROL_REGS
^~~~~~~~~~~~~~~~~~~
/home/frank/runtime/src/coreclr/pal/src/thread/context.cpp:141:9: note: expanded from macro 'ASSIGN_CONTROL_REGS'
ASSIGN_REG(Pc)
^~~~~~~~~~~~~~
/home/frank/runtime/src/coreclr/pal/src/thread/context.cpp:483:25: note: expanded from macro 'ASSIGN_REG'
#define ASSIGN_REG(reg) BSDREG_##reg(ptrace_registers) = lpContext->reg;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<scratch space>:385:1: note: expanded from here
BSDREG_Pc
^
/home/frank/runtime/src/coreclr/pal/src/include/pal/context.h:1020:46: note: expanded from macro 'BSDREG_Pc'
#define BSDREG_Pc(reg) BSD_REGS_STYLE(reg,Pc,pc)
~~~~~~~~~~~~~~~~~~~~~~^~~
/home/frank/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug/pal/src/config.h:158:44: note: expanded from macro 'BSD_REGS_STYLE'
#define BSD_REGS_STYLE(reg, RR, rr) ((reg).rr)
~~~~~ ^
/home/frank/runtime/src/coreclr/pal/src/thread/context.cpp:600:9: error: unknown type name 'fpsimd_context'
fpsimd_context* fp = GetNativeSigSimdContext(native);
^
/home/frank/runtime/src/coreclr/pal/src/thread/context.cpp:744:15: error: unknown type name 'fpsimd_context'
const fpsimd_context* fp = GetConstNativeSigSimdContext(native);
^
4 errors generated.
(Note: using Ubuntu 18.04 with FreeBSD AARCH64 crossroot)
Ok, sounds like again elr vs pc. So can you please make this change?:
#define BSDREG_Pc(reg) BSD_REGS_STYLE(reg,Elr,elr)
Last thing is the floating point context. In context.cpp at line 743, we'll need to add before the #else
:
#elif TARGET_FREEBSD
const struct fpregs* fp = GetConstNativeSigSimdContext(native);
if (fp)
{
lpContext->Fpsr = fp->fp_sr;
lpContext->Fpcr = fp->fp_cr;
for (int i = 0; i < 32; i++)
{
lpContext->V[i] = *(NEON128*) &fp->q[i];
}
}
Similarly at line 599, we need to add before the #else
:
#elif TARGET_FREEBSD
struct fpregs* fp = GetNativeSigSimdContext(native);
if (fp)
{
fp->fp_sr = lpContext->Fpsr;
fp->fp_cr = lpContext->Fpcr;
for (int i = 0; i < 32; i++)
{
*(NEON128*) &fp->q[i] = lpContext->V[i];
}
}
Including above code:
/home/frank/runtime/src/coreclr/pal/src/thread/context.cpp:607:34: error: no member named 'q' in 'fpregs'
*(NEON128*) &fp->q[i] = lpContext->V[i];
~~ ^
/home/frank/runtime/src/coreclr/pal/src/thread/context.cpp:762:52: error: no member named 'q' in 'fpregs'
lpContext->V[i] = *(NEON128*) &fp->q[i];
~~ ^
2 errors generated.
Not sure why its throwing that. usr/include/machine/ucontext.h
on the crossrootfs has that member and seen (https://github.com/freebsd/freebsd-src/blob/373ffc62c158e52cde86a5b934ab4a51307f9f2e/sys/arm64/include/ucontext.h#L45)
@janvorli Big thanks for help/pointers. I was able to compile coreclr part (there were some typos to fix):
Repo successfully built.
Product binaries are available at /usr/home/sec/runtime/artifacts/bin/coreclr/FreeBSD.arm64.Debug
Also I've pushed those changes here https://github.com/dotnet/runtime/compare/main...sec:runtime:freebsd.arm64 Any way to test if all those register parts are working correctly (aka how to run the tests for the part that just got compiled) ?
@Thefrank I also took few changes from your patch regarding RID's - thanks :) Would you be able to try to cross compile under Linux? I've tried, but on my setup (Linux ARM64) it fails with:
Scanning dependencies of target coreclrpal
[ 1%] Building CXX object pal/src/CMakeFiles/coreclrpal.dir/cruntime/file.cpp.o
[ 1%] Building C object libs-native/System.Native/CMakeFiles/System.Native-Static.dir/pal_tcpstate.c.o
In file included from /home/ubuntu/runtime/src/coreclr/pal/src/cruntime/file.cpp:21:
/home/ubuntu/runtime/src/coreclr/pal/src/include/pal/palinternal.h:158:10: fatal error: 'type_traits' file not found
#include <type_traits>
^~~~~~~~~~~~~
[ 1%] Building C object libs-native/System.IO.Compression.Native/CMakeFiles/System.IO.Compression.Native-Static.dir/home/ubuntu/runtime/src/native/external/brotli/enc/backward_references.c.o
[ 1%] Building C object libs-native/System.Native/CMakeFiles/System.Native-Static.dir/pal_threading.c.o
1 error generated.
make[2]: *** [pal/src/CMakeFiles/coreclrpal.dir/build.make:76: pal/src/CMakeFiles/coreclrpal.dir/cruntime/file.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:2020: pal/src/CMakeFiles/coreclrpal.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
Also if you'll take my branch, those can also be needed - don't know if those are needed or it's just my setup:
diff --git a/eng/common/cross/toolchain.cmake b/eng/common/cross/toolchain.cmake
index d5dfc13504b..478029410b7 100644
--- a/eng/common/cross/toolchain.cmake
+++ b/eng/common/cross/toolchain.cmake
@@ -133,7 +133,11 @@ elseif(FREEBSD)
set(CMAKE_CXX_COMPILER_TARGET ${triple})
set(CMAKE_ASM_COMPILER_TARGET ${triple})
set(CMAKE_SYSROOT "${CROSS_ROOTFS}")
- set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fuse-ld=lld")
+ if(CLR_CMAKE_HOST_OS STREQUAL FreeBSD)
+ set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fuse-ld=lld")
+ else()
+ set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fuse-ld=lld -stdlib=libc++ -Qunused-arguments")
+ endif()
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -fuse-ld=lld")
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} -fuse-ld=lld")
elseif(ILLUMOS)
diff --git a/eng/native/configuretools.cmake b/eng/native/configuretools.cmake
index 6697524c659..b44bcc3417c 100644
--- a/eng/native/configuretools.cmake
+++ b/eng/native/configuretools.cmake
@@ -54,7 +54,11 @@ if(NOT WIN32 AND NOT CLR_CMAKE_TARGET_BROWSER)
set(TOOLSET_PREFIX ${ANDROID_TOOLCHAIN_PREFIX})
elseif(CMAKE_CROSSCOMPILING AND NOT DEFINED CLR_CROSS_COMPONENTS_BUILD AND
CMAKE_SYSTEM_PROCESSOR MATCHES "^(armv8l|armv7l|armv6l|aarch64|arm|s390x|ppc64le)$")
- set(TOOLSET_PREFIX "${TOOLCHAIN}-")
+ if(FREEBSD)
+ set(TOOLSET_PREFIX "llvm-")
+ else()
+ set(TOOLSET_PREFIX "${TOOLCHAIN}-")
+ endif()
else()
set(TOOLSET_PREFIX "")
endif()
What should be next steps to get full SDK - as under FreeBSD ARM64 I don't have bootstrapped one, I think I will move to AMD64 FreeBSD and try there to cross compile?
edit: @Thefrank for the error, it's fp_q
and also sorry I wrongly tagged @wfurt :)
edit 2: cross build under FreeBSD is going well... does this mean I will have working SDK on FreeBSD-arm64? 🤞
@sec I was happy to help. As for testing whether these changes are correct, running coreclr tests is the best way. These code paths are exercised by exception handling.
On the target, you can build all coreclr tests (in checked
flavor, which means mostly optimized code, but including asserts in PAL / runtime) using
src/tests/build.sh checked -priority1
(Please note that you have to have the runtime clr+libs part built there before building the tests.)
And then run them using
src/tests/run.sh checked
Hm, not that easy @janvorli :) While doing crosscompile under FreeBSD-amd64, using command
ROOTFS_DIR=/home/sec/arm64_rootfs eng/build.sh -ci -c Debug -cross -os FreeBSD -a arm64 /p:IsEligibleForNgenOptimization=false -subset Clr.Native+Host.Native
it fails after soem time:
Repo successfully built.
Product binaries are available at /usr/home/sec/runtime/artifacts/bin/coreclr/FreeBSD.arm64.Debug
Executing "/usr/home/sec/runtime/src/coreclr/build-runtime.sh" -arm64 -cmakeargs "-DCLR_CROSS_COMPONENTS_BUILD=1" -debug -ci -os FreeBSD -hostarch x64 -component crosscomponents
Commencing CoreCLR Repo build
__DistroRid: freebsd-arm64
Setting up directories for build
Checking prerequisites...
Commencing build of " crosscomponents " target in "CoreCLR component" for FreeBSD.arm64.Debug in /usr/home/sec/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug/x64
Invoking "/usr/home/sec/runtime/eng/native/gen-buildsys.sh" "/usr/home/sec/runtime/src/coreclr" "/usr/home/sec/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug/x64" x64 clang Debug "" -DCLR_CMAKE_TARGET_ARCH=arm64 -DCLR_CMAKE_PGO_INSTRUMENT=0 -DCLR_CMAKE_OPTDATA_PATH= -DCLR_CMAKE_PGO_OPTIMIZE=0 -DFEATURE_DISTRO_AGNOSTIC_SSL=1 -DCLR_CROSS_COMPONENTS_BUILD=1
/usr/home/sec/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug/x64 /usr/home/sec/runtime/src/coreclr
Not searching for unused variables given on the command line.
-- The C compiler identification is Clang 10.0.1
-- The CXX compiler identification is Clang 10.0.1
...
[ 2%] Linking CXX static library libgcinfo_universal_arm64.a
In file included from /usr/home/sec/runtime/src/coreclr/pal/src/exception/seh.cpp:407:
In file included from /usr/home/sec/runtime/src/coreclr/pal/src/exception/seh-unwind.cpp:34:
/home/sec/arm64_rootfs/usr/local/include/libunwind.h:25:11: fatal error: 'libunwind-x86_64.h' file not found
# include "libunwind-x86_64.h"
^~~~~~~~~~~~~~~~~~~~
[ 2%] Built target gcinfo_universal_arm64
[ 2%] Building CXX object pal/src/CMakeFiles/coreclrpal.dir/file/directory.cpp.o
1 error generated.
--- pal/src/CMakeFiles/coreclrpal.dir/exception/seh.cpp.o ---
*** [pal/src/CMakeFiles/coreclrpal.dir/exception/seh.cpp.o] Error code 1
make[3]: stopped in /usr/home/sec/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug/x64
--- CMakeFiles/crosscomponents.dir/rule ---
*** [CMakeFiles/crosscomponents.dir/rule] Error code 6
make[1]: stopped in /usr/home/sec/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug/x64
1 error
make[1]: stopped in /usr/home/sec/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug/x64
*** [crosscomponents] Error code 2
make: stopped in /usr/home/sec/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug/x64
make: stopped in /usr/home/sec/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug/x64
1 error
/usr/home/sec/runtime/src/coreclr
What's going on in here? @Thefrank how's your Linux crossbuild?
The cross components that are failing to build are compiled for x64, so they should be built on the host without the rootfs usage. It seems like there is some problem causing that the rootfs is still being used.
So for my Ubuntu 18.04 AMD64 -> FreeBSD AARCH64 an old bug has presented itself:
[edit: ROOTFS_DIR=/home/frank/freebsdarm/ ./src/coreclr/build-runtime.sh -arm64 --clang11 -cross -os FreeBSD
]
/home/frank/runtime/src/coreclr/debug/daccess/daccess.cpp:6996:38: error: invalid suffix 'x' on integer constant
*dacTableAddress = baseAddress + DAC_TABLE_RVA;
^
/home/frank/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug/inc/dactablerva.h:1:24: note: expanded from macro 'DAC_TABLE_RVA'
#define DAC_TABLE_RVA 0x c06bb8
^
1 error generated.
Previously under net6, this could be worked around in eng/native/gen-buildsys.sh
diff --git a/eng/native/gen-buildsys.sh b/eng/native/gen-buildsys.sh
index bf04c26f2b1..7d8793dd92c 100755
--- a/eng/native/gen-buildsys.sh
+++ b/eng/native/gen-buildsys.sh
@@ -97,7 +97,7 @@ if [[ "$build_arch" == "wasm" ]]; then
cmake_command="emcmake $cmake_command"
fi
-cmake_args_to_cache="$scan_build\n$SCAN_BUILD_COMMAND\n$generator\n$__UnprocessedCMakeArgs"
+cmake_args_to_cache=$(printf "$scan_build${SCAN_BUILD_COMMAND:+ }${SCAN_BUILD_COMMAND} $generator $__UnprocessedCMakeArgs")
cmake_args_cache_file="$2/cmake_cmd_line.txt"
if [[ -z "$__ConfigureOnly" ]]; then
if [[ -e "$cmake_args_cache_file" ]]; then
This whole codeblock was removed in https://github.com/dotnet/runtime/commit/d22aceff4b4b83672cfaaab92d9977445213f9f5 so this hacky-workaround won't work.
The contents of the dactablerva.h is generated by this script: https://github.com/dotnet/runtime/blob/main/src/coreclr/pal/tools/gen-dactable-rva.sh Sounds like this generates the extra spaces, so we can fix it there: https://github.com/dotnet/runtime/blob/879f201ceff095ff59a1f6d2373cee871d7aa102/src/coreclr/pal/tools/gen-dactable-rva.sh#L18
@janvorli About that crosscomponents compile issue... Looking at the command "/usr/home/sec/runtime/src/coreclr/build-runtime.sh" -arm64 -cmakeargs "-DCLR_CROSS_COMPONENTS_BUILD=1" -debug -ci -os FreeBSD -hostarch x64 -component crosscomponents
- there's no -cross
so it shouldn't use cross fs, but it's using, so looks like it's env thing - anyway - if I run the command directly it fail with something more harder to fix (hope not):
Consolidate compiler generated dependencies of target cee_wks_obj
[ 21%] Building ASM object vm/wks/CMakeFiles/cee_wks_core.dir/__/arm64/asmhelpers.S.o
[ 21%] Building CXX object vm/wks/CMakeFiles/cee_wks_obj.dir/__/codeman.cpp.o
[ 21%] Building CXX object pal/src/CMakeFiles/coreclrpal.dir/map/map.cpp.o
/usr/home/sec/runtime/src/coreclr/vm/arm64/asmhelpers.S:9:5: error: unknown use of instruction mnemonic without a size suffix
mov x0, lr
^
/usr/home/sec/runtime/src/coreclr/vm/arm64/asmhelpers.S:10:5: error: unknown use of instruction mnemonic without a size suffix
ret lr
^
/usr/home/sec/runtime/src/coreclr/vm/arm64/asmhelpers.S:15:5: error: unknown use of instruction mnemonic without a size suffix
mov x0, sp
^
/usr/home/sec/runtime/src/coreclr/vm/arm64/asmhelpers.S:16:5: error: unknown use of instruction mnemonic without a size suffix
ret lr
^
/usr/home/sec/runtime/src/coreclr/vm/arm64/asmhelpers.S:21:5: error: invalid instruction mnemonic 'mrs'
mrs x0, dczid_el0
^~~
/usr/home/sec/runtime/src/coreclr/vm/arm64/asmhelpers.S:22:5: error: unknown use of instruction mnemonic without a size suffix
and x0, x0, 31
^
/usr/home/sec/runtime/src/coreclr/vm/arm64/asmhelpers.S:23:5: error: unknown use of instruction mnemonic without a size suffix
ret lr
^
/usr/home/sec/runtime/src/coreclr/vm/arm64/asmhelpers.S:32:13: error: unknown token in expression
mov w1, #0
^
/usr/home/sec/runtime/src/coreclr/vm/arm64/asmhelpers.S:33:16: error: expected ']' in brackets expression
str w1, [x0, #0xc0]
^
/usr/home/sec/runtime/src/coreclr/vm/arm64/asmhelpers.S:35:16: error: expected ']' in brackets expression
str lr, [x0, #((0xc0 +8)+8)]
^
/usr/home/sec/runtime/src/coreclr/vm/arm64/asmhelpers.S:38:5: error: unknown use of instruction mnemonic without a size suffix
mov x1, sp
^
/usr/home/sec/runtime/src/coreclr/vm/arm64/asmhelpers.S:39:16: error: expected ']' in brackets expression
str x1, [x0, #(0xc0 +8)]
^
/usr/home/sec/runtime/src/coreclr/vm/arm64/asmhelpers.S:42:17: error: unknown token in expression
add x1, x0, #0x0
... a lot more of those errors ...
edit: running ./src/coreclr/build-runtime.sh -arm64 -hostarch arm64 -component crosscomponents -cmakeargs -DCLR_CROSS_COMPONENTS_BUILD=1
on FreeBSD-arm64 fail with
Consolidate compiler generated dependencies of target coreclr
[ 94%] Linking CXX shared library libcoreclr.so
ld.lld: error: unable to find library -lSystem.Globalization.Native-Static
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
--- dlls/mscoree/coreclr/libcoreclr.so ---
*** [dlls/mscoree/coreclr/libcoreclr.so] Error code 1
Albo native libs compile fine under ARM64 host, but executing ./src/coreclr/build-runtime.sh
fail with:
[ 94%] Linking CXX shared library libcoreclr.so
ld.lld: error: unable to find library -lSystem.Globalization.Native-Static
[ 94%] Building CXX object jit/CMakeFiles/clrjit_win_x86_arm64.dir/typeinfo.cpp.o
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
and
sec@amper:~/runtime % ls artifacts/bin/native/FreeBSD-arm64-Debug/
./ libSystem.IO.Compression.Native.so.dbg libSystem.Net.Security.Native.a
../ libSystem.IO.Ports.Native.a libSystem.Net.Security.Native.so*
libSystem.Globalization.Native.a libSystem.IO.Ports.Native.so* libSystem.Net.Security.Native.so.dbg
libSystem.Globalization.Native.so* libSystem.IO.Ports.Native.so.dbg libSystem.Security.Cryptography.Native.OpenSsl.a
libSystem.Globalization.Native.so.dbg libSystem.Native.a libSystem.Security.Cryptography.Native.OpenSsl.so*
libSystem.IO.Compression.Native.a libSystem.Native.so* libSystem.Security.Cryptography.Native.OpenSsl.so.dbg
libSystem.IO.Compression.Native.so* libSystem.Native.so.dbg
edit: trying to build those components under Linux is working fine - maybe this is something wrong with cmakefiles def's ? :/
Tried to use Linux for crosscompile
eng/common/cross/toolchain.cmake
need a fix when setting FreeBSD triple x86_64 -> aarch64but even then it fails, at mono stage, so that's the far most progress:
[31/351] Building C object mono/mini/CMakeFiles/debugger-objects.dir/__/component/debugger-agent.c.o
FAILED: mono/mini/CMakeFiles/debugger-objects.dir/__/component/debugger-agent.c.o
/usr/bin/clang-10 --target=aarch64-unknown-freebsd12 --sysroot=/home/linux/arm64/ -DHAVE_CONFIG_H -I/home/linux/runtime/artifacts/obj -I/home/linux/runtime/src/native -Imono/mini -Imono/mini/../.. -Imono/mini/../../mono/eglib -I/home/linux/runtime/src/mono/mono/mini/../.. -I/home/linux/runtime/src/mono/mono/mini/.. -I/home/linux/runtime/src/mono/mono/mini/../eglib -I/home/linux/runtime/src/mono/mono/mini/../sgen -I/home/linux/runtime/src/mono/mono/mini/../eventpipe -Imono/mini/eventpipe -I/home/linux/runtime/src/native/public/. -std=gnu99 -fno-strict-aliasing -fwrapv -Wall -Wunused -Wmissing-declarations -Wpointer-arith -Wno-cast-qual -Wwrite-strings -Wno-switch -Wno-switch-enum -Wno-unused-value -Wno-attributes -Wno-format-zero-length -Wno-unused-function -Qunused-arguments -Wno-tautological-compare -Wno-parentheses-equality -Wno-self-assign -Wno-return-stack-address -Wno-constant-logical-operand -Wno-zero-length-array -Wno-asm-operand-widths -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Werror=return-type -Werror=implicit-function-declaration -Werror=incompatible-pointer-types -g -g -fPIC -fvisibility=hidden -ffp-contract=off -MD -MT mono/mini/CMakeFiles/debugger-objects.dir/__/component/debugger-agent.c.o -MF mono/mini/CMakeFiles/debugger-objects.dir/__/component/debugger-agent.c.o.d -o mono/mini/CMakeFiles/debugger-objects.dir/__/component/debugger-agent.c.o -c /home/linux/runtime/src/mono/mono/component/debugger-agent.c
<inline asm>:1:6: error: invalid token in expression
int $03
^
<inline asm>:1:6: error: invalid operand
int $03
^
2 errors generated.
ninja: build stopped: subcommand failed.
/home/linux/runtime/src/mono/mono.proj(541,5): error MSB3073: The command "TARGET_BUILD_ARCH=arm64 PKG_CONFIG_PATH=/home/linux/arm64//usr/lib/aarch64-linux-gnu/pkgconfig TARGET_BUILD_ARCH=x64 cmake --build . --target install --config Debug" exited with code 1.
##vso[task.logissue type=error;sourcepath=/home/linux/runtime/src/mono/mono.proj;linenumber=541;columnnumber=5;code=MSB3073;](NETCORE_ENGINEERING_TELEMETRY=Build) The command "TARGET_BUILD_ARCH=arm64 PKG_CONFIG_PATH=/home/linux/arm64//usr/lib/aarch64-linux-gnu/pkgconfig TARGET_BUILD_ARCH=x64 cmake --build . --target install --config Debug" exited with code 1.
Build FAILED.
/home/linux/runtime/src/mono/mono.proj(541,5): error MSB3073: The command "TARGET_BUILD_ARCH=arm64 PKG_CONFIG_PATH=/home/linux/arm64//usr/lib/aarch64-linux-gnu/pkgconfig TARGET_BUILD_ARCH=x64 cmake --build . --target install --config Debug" exited with code 1.
0 Warning(s)
1 Error(s)
Commands I used:
ROOTFS_DIR=/home/linux/arm64/ ./eng/build.sh -ci -c Debug -cross -os FreeBSD -a arm64 /p:IsEligibleForNgenOptimization=false -subset Clr.Native+Host.Native
- compile fineROOTFS_DIR=/home/linux/arm64/ ./eng/build.sh -ci -c Debug -cross -os FreeBSD -a arm64 /p:IsEligibleForNgenOptimization=false /p:AppHostSourcePath=/home/linux/runtime/artifacts/obj/freebsd-arm64.Debug/apphost/standalone/apphost
- failed
- RVA is caused I think when you have bash version <5.1 - dirty hack was to remove 016 from the format string in script
frank@buildbot:~/runtime$ bash --version GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)
O.K. /sigh/ removing that deleting the old
artifacts/obj/coreclr/FreeBSD.arm64.Debug/inc/dactablerva.h
and rerunning finished! the new file contained#define DAC_TABLE_RVA 0xc06bb8
Repo successfully built. Product binaries are available at /home/frank/runtime/artifacts/bin/coreclr/FreeBSD.arm64.Debug
It also makes it past the cross components but runs into the same issue for mono.
Which is interesting because...FreeBSD doesn't have pkg-config as a part of the base OS. It's in ports (as pkgconf
). That path looks like it gets added from the mono.proj
TOP EDIT: almost forgot to give credit to @janvorli for all the help It's built!
Build succeeded.
0 Warning(s)
0 Error(s)
Time Elapsed 00:43:38.40
I just have no way to test it as...I don't have any ARM64 hardware sitting around. Posting patch + binaries shortly
EDIT1: freebsd_arm_patch.txt
EDIT2: https://github.com/Thefrank/dotnet-freebsd-crossbuild/releases/tag/V7.0-ARM64-EXPERIMENTAL
EDIT3: build command ROOTFS_DIR=/home/frank/freebsd13arm/ ./build.sh -a arm64 --clang11 -cross -os FreeBSD
EDIT4: Tomorrow ill look into ASPNET and Installer, both of which should be much easier than runtime
@Thefrank @janvorli
I was wrong about RVa issue - I have same version of bash and it format diffrently - but made a fix for that - https://github.com/sec/runtime/commit/1a6d28850f973de8a0713e785996f656389b27f8
Also all the other fixes (with some not needed removed) are integrated into my branch - would it be feasible to make PR for that at this stage @wfurt ?
Thanks to last mono fix - I was also able to crosscompile Linux x64 -> FreeBSD arm64
Linux arm64 -> FreeBSD arm64 - also fine, with small patch (edit: looks it's no longer need, maybe becuase clang11 was used, but I will leave it here for future reference anyway):
diff --git a/eng/common/cross/toolchain.cmake b/eng/common/cross/toolchain.cmake
index d5dfc13504b..478029410b7 100644
--- a/eng/common/cross/toolchain.cmake
+++ b/eng/common/cross/toolchain.cmake
@@ -133,7 +133,11 @@ elseif(FREEBSD)
set(CMAKE_CXX_COMPILER_TARGET ${triple})
set(CMAKE_ASM_COMPILER_TARGET ${triple})
set(CMAKE_SYSROOT "${CROSS_ROOTFS}")
- set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fuse-ld=lld")
+ if(CLR_CMAKE_HOST_OS STREQUAL FreeBSD)
+ set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fuse-ld=lld")
+ else()
+ set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fuse-ld=lld -stdlib=libc++ -Qunused-arguments")
+ endif()
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -fuse-ld=lld")
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} -fuse-ld=lld")
elseif(ILLUMOS)
FreeBSD x64 -> FreeBSD arm64 - two issues:
crosscomponents
- looks like crossrootfs is used/tmp/asmhelpers-8f0fb3.s
(more above)FreeBSD arm64:
./src/coreclr/build-runtime.sh
- ok./src/native/libs/build-native.sh
- okNOW, testing runtime under FreeBSD arm64 host...
[sec@amper ~]$ ~/dotnet7/dotnet --info
Host:
Version: 7.0.0-dev
Architecture: arm64
Commit: N/A
.NET SDKs installed:
No SDKs were found.
.NET runtimes installed:
Microsoft.NETCore.App 7.0.0-dev [/usr/home/sec/dotnet7/shared/Microsoft.NETCore.App]
Other architectures found:
None
Environment variables:
Not set
global.json file:
Not found
Learn more:
https://aka.ms/dotnet/info
Download .NET:
https://aka.ms/dotnet/download
Tried to make simple console app, restore, build, publish, copy to arm64 host, adjust json to use 7.0.0-dev and
[sec@amper ~/runtime/console_app]$ ~/dotnet7/dotnet bin/Debug/net6.0/publish/console_app.dll
Failed to create CoreCLR, HRESULT: 0x80004005
Attaching COREHOST trace file - hope it's something trivial to fix :) host.txt
yes, I think so @sec. main branch for 7.0 is still generally open (closing soon). I'm on the road so my capacity is small for next 2-3 weeks.
@sec if run in jail, does the jail have allow_mlock
on? Does the jail/OS have a defined and writable TMP folder?
Not running in jail and yes, there's a writeable /tmp folder.
@janvorli any idea on how to debug this problem?
truss
work under ARM64? Might give that a shot
@Thefrank tried that already, nothing special, but attaching sample log truss_output.txt
attempt at full SDK. Still no way to test it:https://github.com/Thefrank/dotnet-freebsd-crossbuild/releases/tag/7.0.100-preview.7.22353.99-freebsd-arm64
@Thefrank nice - tried SDK, same result (one thing I notice is no runtime info about FreeBSD host etc., but that's why it fail right?):
sec@amper:~/sdk7 % ./dotnet --info
Failed to create CoreCLR, HRESULT: 0x80004005
Host:
Version: 7.0.0-preview.6.22329.5
Architecture: arm64
Commit: c9140d621a
.NET SDKs installed:
7.0.100-preview.7.22353.99 [/usr/home/sec/sdk7/sdk]
.NET runtimes installed:
Microsoft.AspNetCore.App 7.0.0-preview.7.22351.1 [/usr/home/sec/sdk7/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 7.0.0-preview.6.22329.5 [/usr/home/sec/sdk7/shared/Microsoft.NETCore.App]
Other architectures found:
None
Environment variables:
Not set
global.json file:
Not found
Learn more:
https://aka.ms/dotnet/info
Download .NET:
https://aka.ms/dotnet/download
Output from SDK7 under x64 host:
.NET SDK:
Version: 7.0.100-preview.5.22307.18
Commit: bd8b088037
Runtime Environment:
OS Name: FreeBSD
OS Version: 13
OS Platform: FreeBSD
RID: freebsd.13-x64
Base Path: /usr/home/sec/dotnet7/sdk/7.0.100-preview.5.22307.18/
global.json file:
Not found
Host:
Version: 7.0.0-preview.5.22301.12
Architecture: x64
Commit: 425fedc0fb
.NET SDKs installed:
7.0.100-preview.5.22307.18 [/usr/home/sec/dotnet7/sdk]
.NET runtimes installed:
Microsoft.AspNetCore.App 7.0.0-preview.5.22303.8 [/usr/home/sec/dotnet7/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 7.0.0-preview.5.22301.12 [/usr/home/sec/dotnet7/shared/Microsoft.NETCore.App]
@Thefrank could you share the steps/any patches that you used to make this - maybe we could use this to automate x64 builds and publish them on https://github.com/dotnet/runtimelab ?
Does anyone have any expierence in troubleshoting root cuase of this CLR fail - maybe something from https://github.com/dotnet/diagnostics will be to good use?
@Thefrank nice - tried SDK, same result (one thing I notice is no runtime info about FreeBSD host etc., but that's why it fail right?):
From what was posted it does not appear to be making it very far. Now WHY that is happening is beyond me
@Thefrank could you share the steps/any patches that you used to make this - maybe we could use this to automate x64 builds and publish them on https://github.com/dotnet/runtimelab ?
Runtime builds from head with only a quick sed
patch to make it use a prebuilt SDK and that is how I do nightlies+tests for FreeBSD-x64
Building a WHOLE SDK requires a bit more tweaks but is rather easy from TAG.
However, if you want to build from a random commit, well, that takes a bit more work. For that I usually work backwards; examine what BUILDIDs dotnet/installer wants for dotnet/runtime and dotnet/aspnetcore, reverse them into YYYYMMDD.# format and plug them into the builds first for runtime and second for aspnetcore.
EDIT: Does lldb
provide any insights? Is it even making it to managed code?
@Thefrank I was rather asking, how to you build e. g. installer
repo in cross
mode, as I didn't saw any swithces just for that (and --runtime-id freebsd-x64 /p:OSName=freebsd
prepare SDK with correct names, but Linux inside) - but that's offtopic :)
As for lldb
- not an export on this one, but doesn't look like it's getting into managed code - tried to use corerun
with bogus parameters and it's throwing the same error before even checking if given file exists.
edit: I may be wrong, but maybe it's connected with fact, that cross build from FreeBSD-x64 host fail on cross components stage with some assembler compiler error (but is fine when doing the same from Linux host)?
Had fun with lldb
, learning at 2am is great :) The code looks like it's failing when loaded/loading System.Private.CoreLib
and then going this route:
* thread #1, name = 'corerun', stop reason = breakpoint 7.1
frame #0: 0x0000000041544ce8 libcoreclr.so`Assembly::Init(this=0x0000000040c11650, pamTracker=0x0000ffffffffccf0, pLoaderAllocator=0x0000000000000000) at assembly.cpp:196:53
193 // manifest modules of dynamic assemblies are always transient
194 m_pModule = ReflectionModule::Create(this, pPEAssembly, pamTracker, REFEMIT_MANIFEST_MODULE_NAME);
195 else
-> 196 m_pModule = Module::Create(this, mdFileNil, pPEAssembly, pamTracker);
197
198 InterlockedIncrement((LONG*)&g_cAssemblies);
199
after some steps
Process 25614 stopped
* thread #1, name = 'corerun', stop reason = step over
frame #0: 0x000000004156461c libcoreclr.so`Module::Initialize(this=0x0000000042224000, pamTracker=0x0000ffffffffccf0, szName=0x0000000000000000) at ceeload.cpp:443:63
440
441 #ifdef FEATURE_READYTORUN
442 m_pNativeImage = NULL;
-> 443 if ((m_pReadyToRunInfo = ReadyToRunInfo::Initialize(this, pamTracker)) != NULL)
444 {
445 m_pNativeImage = m_pReadyToRunInfo->GetNativeImage();
446 if (m_pNativeImage != NULL)
(lldb)
BEGIN: coreclr_initialize failed - Error: 0x80004005
time for bed...
edit: I think final fail is here:
Process 26961 stopped
* thread #1, name = 'corerun', stop reason = step over
frame #0: 0x000000004172268c libcoreclr.so`ReadyToRunInfo::Initialize(pModule=0x0000000042254000, pamTracker=0x0000ffffffffccf0) at readytoruninfo.cpp:561:9
558 if (!pLayout->IsNativeMachineFormat())
559 {
560 // For CoreCLR, be strict about disallowing machine mismatches.
-> 561 COMPlusThrowHR(COR_E_BADIMAGEFORMAT);
562 }
Is this something wrong with cross compiled runtime/sdk ?
It's not libcoreclr.so
fault for sure
sec@amper:~/runtime % file artifacts/bin/coreclr/FreeBSD.arm64.Debug/libcoreclr.so
artifacts/bin/coreclr/FreeBSD.arm64.Debug/libcoreclr.so: ELF 64-bit LSB shared object, ARM aarch64, version 1 (FreeBSD), dynamically linked, for FreeBSD 13.1, BuildID[sha1]=3d30f6f528036a298d342a15af8b3b1e47af6a09, stripped
sec@amper:~/runtime % ldd artifacts/bin/coreclr/FreeBSD.arm64.Debug/libcoreclr.so
artifacts/bin/coreclr/FreeBSD.arm64.Debug/libcoreclr.so:
libthr.so.3 => /lib/libthr.so.3 (0x4195e000)
librt.so.1 => /usr/lib/librt.so.1 (0x419ba000)
libunwind-aarch64.so.8 => /usr/local/lib/libunwind-aarch64.so.8 (0x419ef000)
libunwind.so.8 => /usr/local/lib/libunwind.so.8 (0x41a51000)
libc++.so.1 => /usr/lib/libc++.so.1 (0x41aae000)
libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x41ba9000)
libm.so.5 => /lib/libm.so.5 (0x41bf5000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x41c63000)
libc.so.7 => /lib/libc.so.7 (0x401e5000)
liblzma.so.5 => /usr/lib/liblzma.so.5 (0x41ccc000)
libz.so.6 => /lib/libz.so.6 (0x41d21000)
libmd.so.6 => /lib/libmd.so.6 (0x41d68000)
So it's this one that's faulty?
sec@amper:~/runtime % file artifacts/bin/coreclr/FreeBSD.arm64.Debug/System.Private.CoreLib.dll
artifacts/bin/coreclr/FreeBSD.arm64.Debug/System.Private.CoreLib.dll: PE32+ executable (DLL) (console) Mono/.Net assembly, for MS Windows
When doing cross build Linux -> FreeBSD arm64, is this ok on Linux host - shouldn't this be "freebsd-arm64" by any chance?
./artifacts/bin/coreclr/FreeBSD.arm64.Debug/crossgen2/linux-arm64/System.Private.CoreLib.dll
./artifacts/bin/coreclr/FreeBSD.arm64.Debug/crossgen2/linux-arm64/publish/System.Private.CoreLib.dll
Checking Microsoft.NETCore.App.Crossgen2.freebsd-arm64.7.0.0-dev.nupkg
there are Linux binaries inside:
sec@amper:~/newb/tools % file crossgen2
crossgen2: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=725302cc1678f04d8935457644ebd9deeb09f8ab, stripped
sec@amper:~/newb/tools % file dotnet
dotnet: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=eb185bf4cb8984561b845c57ea43484c3ed2b40a, stripped
sec@amper:~/newb/tools % file corerun
corerun: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 13.0 (1300139), FreeBSD-style, BuildID[sha1]=da7462690d3094983f41df3114505547a61835c8, with debug_info, not stripped
@Thefrank @janvorli @jkotas Maybe this is silly, but managed to get runtime/SDK to work (at some point at least). What I did was
System.Private.CoreLib.dll
from cross compiled arm64 runtime/SDKtools/crossgen2 System.Private.CoreLib.dll --targetarch arm64 --out System.Private.CoreLib.dll.2
on itAnd
sec@amper:~/sdk7 % ./dotnet --info
.NET SDK:
Version: 7.0.100-preview.7.22353.99
Commit: be72bcc21b
Runtime Environment:
OS Name: FreeBSD
OS Version: 13
OS Platform: FreeBSD
RID: freebsd.13-arm64
Base Path: /usr/home/sec/sdk7/sdk/7.0.100-preview.7.22353.99/
Host:
Version: 7.0.0-preview.6.22329.5
Architecture: arm64
Commit: c9140d621a
.NET SDKs installed:
7.0.100-preview.7.22353.99 [/usr/home/sec/sdk7/sdk]
.NET runtimes installed:
Microsoft.AspNetCore.App 7.0.0-preview.7.22351.1 [/usr/home/sec/sdk7/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 7.0.0-preview.6.22329.5 [/usr/home/sec/sdk7/shared/Microsoft.NETCore.App]
Run simple console app published from Windows host:
sec@amper:~/dotnet7 % ./dotnet ~/runtime/console_app/bin/Debug/net6.0/publish/console_app.dll
Hello, World!
But:
sec@amper:~/temp % ~/sdk7/dotnet new console
Segmentation fault (core dumped)
More info:
sec@amper:~/temp % lldb14 ~/sdk7/dotnet new console
(lldb) target create "/home/sec/sdk7/dotnet"
Current executable set to '/home/sec/sdk7/dotnet' (aarch64).
(lldb) settings set -- target.run-args "new" "console"
(lldb) r
Process 27236 launched: '/home/sec/sdk7/dotnet' (aarch64)
Process 27236 stopped
* thread #1, name = 'dotnet', stop reason = signal SIGSEGV: invalid address (fault address: 0x0)
frame #0: 0x0000000041859f6c libunwind.so.8`___lldb_unnamed_symbol150 + 32
libunwind.so.8`___lldb_unnamed_symbol150:
-> 0x41859f6c <+32>: ldr x8, [x1]
0x41859f70 <+36>: mov w0, wzr
0x41859f74 <+40>: str x8, [x2]
0x41859f78 <+44>: ldr x19, [sp, #0x20]
(lldb) bt
* thread #1, name = 'dotnet', stop reason = signal SIGSEGV: invalid address (fault address: 0x0)
* frame #0: 0x0000000041859f6c libunwind.so.8`___lldb_unnamed_symbol150 + 32
frame #1: 0x000000004185af70 libunwind.so.8`_ULaarch64_step + 568
frame #2: 0x000000004178e8ec libcoreclr.so`___lldb_unnamed_symbol14794 + 256
frame #3: 0x00000000415b29ec libcoreclr.so`___lldb_unnamed_symbol7901 + 256
frame #4: 0x00000000413f75a0 libcoreclr.so`___lldb_unnamed_symbol2153 + 136
frame #5: 0x00000000413f7508 libcoreclr.so`___lldb_unnamed_symbol2152 + 28
frame #6: 0x00000000415b8400 libcoreclr.so`___lldb_unnamed_symbol8017 + 84
frame #7: 0x00000000415b7db0 libcoreclr.so`___lldb_unnamed_symbol8013 + 752
frame #8: 0x00000000415b7198 libcoreclr.so`___lldb_unnamed_symbol8011 + 1108
frame #9: 0x00000000415bb444 libcoreclr.so`___lldb_unnamed_symbol8050 + 336
frame #10: 0x00000000415bb71c libcoreclr.so`___lldb_unnamed_symbol8051 + 68
frame #11: 0x0000000041518964 libcoreclr.so`___lldb_unnamed_symbol6091 + 540
@Thefrank makes sense to build SDK in Debug to see where/why it's SIGSEGV or this is easy to fix now? :)
Crossgen2 packages, AFAIK, are no longer generated for FreeBSD as that ability got removed somewhere after net7p3. This is not ideal as those packages are needed for native builds (restore requires it)
Crossbuilt net7p5 for FreeBSD x64 with the package added back in looks like it generates incorrectly too:
Shipping/Microsoft.NETCore.App.Crossgen2.freebsd-x64.7.0.0-preview.5.22301.12/tools # ls | xargs file
coreclr_delegates.h: C source, ASCII text
corerun: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 12.3, FreeBSD-style, BuildID[sha1]=ea37edd26732ab0d86c7eff56bfd2dc4b97d7de5, with debug_info, not stripped
crossgen2: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 12.3, FreeBSD-style, BuildID[sha1]=f2abf1f880cce1b944515bb44ccf8778bd1e0d4d, stripped
dotnet: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=342e6dc49a1ef7d664ecf66554bd91f12377deb8, stripped
hostfxr.h: C source, ASCII text
libclrjit_universal_arm_x64.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=5371055af15bb23ddab55f6d56ea05e68c4ec014, stripped
libclrjit_universal_arm64_x64.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=df1c176a5aedfe127ee1097723e5a9dd1ca0c9d6, stripped
libclrjit_unix_x64_x64.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=c0d6411b4d9dc86718543182049e390bebcc75c0, stripped
libclrjit_win_x64_x64.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=c8498128119ad328a63433b0e113bf35a28b67b8, stripped
libclrjit_win_x86_x64.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=e1ef1e6c1d44175ebe3f2dd1ed7a22bae89dd3ea, stripped
libclrjit.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=b3050876013379e1abd2c8ab67a3b649c901550c, stripped
libcoreclr.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=beb05fea73aadf25e83d75f7cb06a638becb31bd, stripped
libhostfxr.a: current ar archive
libhostfxr.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=955d0bd7775d52bc8aba0124aa2e6f7b0e059ae1, stripped
libhostpolicy.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=bd4f445a772be690487d98d63be8aa0d7013dad0, stripped
libjitinterface_x64.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=7aca7857d900236e5fd0925419afd2a1510541f5, stripped
libmscordaccore.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=fae13f6b4f0a887250f6e2f9bbfa986c0b493edd, stripped
libmscordbi.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=e07ae986afb11b697f758994cb3dd069fe6cd801, stripped
libnethost.a: current ar archive
libnethost.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=fb0e1694b1521c04c74755a2210c8b8988298f81, stripped
libSystem.Globalization.Native.a: current ar archive
libSystem.Globalization.Native.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=411160e3995dfc8aa04d8929dc5958ae61e4d6c4, stripped
libSystem.IO.Compression.Native.a: current ar archive
libSystem.IO.Compression.Native.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=e539b4dd805290e3a19c8551ab766cc7d7d0bafd, stripped
libSystem.IO.Ports.Native.a: current ar archive
libSystem.IO.Ports.Native.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=d41c2ed5987fd9f7ffe8f03c853673dc24f8a397, stripped
libSystem.Native.a: current ar archive
libSystem.Native.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=e732c3d8b2d37c4584bcd8470f018a1493394bc0, stripped
libSystem.Net.Security.Native.a: current ar archive
libSystem.Net.Security.Native.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=3ecfe261dd743c77c13bc8ed53a811d1e0c4c6c3, stripped
libSystem.Security.Cryptography.Native.OpenSsl.a: current ar archive
libSystem.Security.Cryptography.Native.OpenSsl.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, BuildID[sha1]=b3f59bd1083874d1c0db7fc528b3c8452d11be21, stripped
nethost.h: C source, ASCII text
singlefilehost: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=7d2abe4df7dfc9c4f875865784d606267bc42c77, stripped,
net6RTM (the last crossbuild from that branch I have done) only contains one executable: crossgen2 and that is correctly FreeBSD
EDIT: did a quick crossbuild from Linux x86_64 -> FreeBSD ARM64. The contents of the Crossgen2 nupkg (and tarball) are just as odd: everything but dotnet
and singlefilehost
are FreeBSD ARM aarch64. Those two are Linux ARM aarch64
Broken System.Private.CoreLib
those make it work OOTB (for runtime part) - https://github.com/dotnet/runtime/pull/71486/commits/951c92afc499fddb19846bfd108bad5f3a086ae8 and https://github.com/dotnet/runtime/pull/71486/commits/98d39cd008c6160eb2feec60fad712d7b24661c1
As for SDK to fail somewhere inside libunwind
- @Thefrank would you mind doing SDK build in Debug
or share the steps I can reproduce to crossbuild it myself (so I won't bug you) ? :)
As for those Linux files inside packages - that's weird, becuase there is FreeBSD version of singlefilehost
and dotnet
produced during build, but the Linux one is being packed inside - all of the below have their correct FreeBSD, but are inside dotnet-crossgen2-7.0.0-dev-freebsd-arm64.tar.gz
:
./singlefilehost: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=000c66b0f0a90e33362b538857a1fe07858d95d2, stripped
./libhostpolicy.so: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[sha1]=f7f923736c5fa480faec666385f8ae7cb6191905, stripped
./libnethost.so: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[sha1]=ec8c97ecfbcf5bb77d4fa8a5fcb3e56cb7bc13e4, stripped
./dotnet: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=eb185bf4cb8984561b845c57ea43484c3ed2b40a, stripped
./libhostfxr.so: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[sha1]=827719e673ae0b16ade2b80907cf2c39e6ae517a, stripped
At least crossgen2
inside is FreeBSD, bad news it doesn't work:
Assert failure(PID 42015 [0x0000a41f], Thread: 105128 [0x19aa8]): pThread
File: /home/ubuntu/runtime/src/coreclr/vm/threads.inl Line: 43
Image: /usr/home/sec/temp/cros/crossgen2
Abort (core dumped)
I don't mind making the SDKs. This also reminds me that I should update my build script (from the above repo) to allow for arm64 builds.
For platforms like FreeBSD, the installer branch is basically a fancy fetch
+sed
+tar
... unless you want to generate native binaries...which we don't under a crossbuild, as afaik, does not work for FreeBSD.
@Thefrank Thanks, but can't find libcoreclr.so.dbg
anywhere :) (my guess it's not packed into SDK) - so the backtrace is still unknown.
Maybe someone more expierenced in this topic could help to debug the issue (something with threads, as crossgen2 fail in there) ? :)
@Thefrank Nope, libcoreclr.so.dbg
is still not there:
sec@amper:~/sdk % tar --list --file dotnet-runtime-7.0.0-preview.6.22329.5-freebsd-arm64.tar.gz|grep libcoreclr
./shared/Microsoft.NETCore.App/7.0.0-preview.6.22329.5/libcoreclr.so
I don't think *.so.dbg
are packed into tar's
I linked the wrong one dotnet-runtime-symbols-freebsd-arm64-7.0.0-preview.6.22329.5.tar.gz needs to say "symbols"
EDIT: @sec on a different note: do you have any suggestions on cheap arm64 hardware that freebsd runs on?
Hi, Not to spam #14537 (as it's already quite big) :) As full native build of either v6 and v7 is possible under FreeBSD-amd64, I went ahead and tried to make the same process work for ARM64.
After getting configure to success, I've tried:
Command I've used
ROOTFS_DIR=/home/ubuntu/arm64 ./src/coreclr/build-runtime.sh -arm64 -debug clang14 -cross -os FreeBSD
. This does look like something missing in configure step, as those headers are present on rootfs_dir and host system:so I guess something during configure stage should be added to look in those directories?
-cross
and without, it fails with the same error:make[2]: stopped in /usr/home/sec/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug *** [all] Error code 6
make: stopped in /usr/home/sec/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug 1 error
make: stopped in /usr/home/sec/runtime/artifacts/obj/coreclr/FreeBSD.arm64.Debug