search

dotnet / core

.NET news, announcements, release notes, and more!
https://dot.net
MIT License
20.82k stars 4.87k forks source link

3.0.100-preview6/7/8 not working on Yocto ARM/ARM64 #2944

Closed chrfin closed 4 years ago

chrfin commented 5 years ago

We use .NET Core on a Variscite DART-MX8M-MINI SoM and it worked fine with 3.0.100-preview5.
With preview 6 .NET Core doesn't work at all, because as soon as a program is startet it gets stuck without any output (also nothing on the serial console) and needs to be killed. Even dotnet --info does not work. We are running Yocto (Linux imx8mm-var-dart 4.14.78-imx8mm+g33611e8 #1 SMP PREEMPT Tue Jun 4 14:04:01 UTC 2019 aarch64 aarch64 aarch64 GNU/Linux) on the board.

I do not know what information I can also provide, but if you need something I'm happy to provide anything you need.

janvorli commented 5 years ago

Would you be able to attach gdb to the hung dotnet process and then print stack trace of thread 1 and share it?

sudo gdb -p PidOfYourHungDotnet
thread select 1
bt
chrfin commented 5 years ago

Here the output:

root@imx8mm-var-dart:~# sudo gdb -p 636 GNU gdb (GDB) 8.0.1 Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "aarch64-poky-linux". Type "show configuration" for configuration details. For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/. For help, type "help". Type "apropos word" to search for commands related to "word". Attaching to process 636 [New LWP 637] [New LWP 638] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/libthread_db.so.1". 0x0000ffffbd382320 in __lll_lock_wait (futex=futex@entry=0xffffbc61a7d8, private=0) at /usr/src/debug/glibc/2.27-r0/git/nptl/lowlevellock.c:43 43 lll_futex_wait (futex, 2, private); / Wait if futex == 2. */ (gdb) thread 1 [Switching to thread 1 (Thread 0xffffbcf99e80 (LWP 636))]

0 0x0000ffffbd382320 in __lll_lock_wait (futex=futex@entry=0xffffbc61a7d8, private=0) at /usr/src/debug/glibc/2.27-r0/git/nptl/lowlevellock.c:43

43 lll_futex_wait (futex, 2, private); / Wait if futex == 2. */ (gdb) bt

0 0x0000ffffbd382320 in __lll_lock_wait (futex=futex@entry=0xffffbc61a7d8, private=0) at /usr/src/debug/glibc/2.27-r0/git/nptl/lowlevellock.c:43

1 0x0000ffffbd37b648 in __GI___pthread_mutex_lock (mutex=0xffffbc61a7d8) at /usr/src/debug/glibc/2.27-r0/git/nptl/pthread_mutex_lock.c:78

2 0x0000ffffbc5bbe68 in ust_lock_nocheck () from /usr/lib/liblttng-ust.so.0

3 0x0000ffffbc5c17d4 in ?? () from /usr/lib/liblttng-ust.so.0

4 0x0000ffffbc60eeb0 in ?? () from /usr/lib/liblttng-ust.so.0

Backtrace stopped: previous frame inner to this frame (corrupt stack?)

carlossanlop commented 5 years ago

@janvorli should this issue be moved to another repo to increase visibility? Maybe dotnet/iot? or should it be moved to core-setup?

chrfin commented 5 years ago

@janvorli Just tried preview 7 with the same result :-(

chrfin commented 5 years ago

@janvorli @carlossanlop Still not working with preview 8 👎 Is there anything we can do to help fix this issue?

carlossanlop commented 5 years ago

@janvorli @joperezr @krwq who can help with this issue? Should I transfer it to another repo for better visibility?

joperezr commented 5 years ago

dotnet/iot is not the right place for this issue. If this should be transfered, the best place for it would be either coreclr repo or the cli repo. I would probably start with coreclr. That said, I think that either @janvorli or @jkotas would probably be the ones that could help here.

krwq commented 5 years ago

@chrfin how are you running dotnet on ARM64? Could you share the exact steps? Do you have TTY allocated when you are printing stuff on the console? (if not sure try running you app inside tmux which ensures TTY)

jkotas commented 5 years ago

This looks like duplicate of https://github.com/dotnet/coreclr/issues/15693. This hang is caused by a incompatibility between glibc and lttng in Yocto images.

To workaround it, you can either:

shadow-cs commented 5 years ago

@jkotas we are facing similar issue on Yocto build of stm32mp1-dk2-mx 4.19.9 Linux on armv7. The last preview we were able to run our app on was 5 (preview 4 is working as well) every newer preview hangs before the app starts. The last line in trace is CoreCLR path = ....

Setting COMPlus_LTTng=0 does fixes the issue.

May I suggest changing the issue name to just ARM since both architectures appear to be affected.

Possibly also related to:

Edit: Deleting the LTTng libraries also fixes the issue (this has been discussed in other issues referenced above). Note that YOCTO might have issues building the image if LTTng is missing, but deleting it manually does work.

chrfin commented 5 years ago

Setting export COMPlus_LTTng=0 did work, but updating lttng-tools to 2.10.6 (previous version was 2.9.5) did not work.

carlossanlop commented 4 years ago

As @jkotas mentioned, this seems like a duplicate of dotnet/coreclr#15693 . Since the workaround did not work, let's please move the discussion to that issue/repo for better visibility.