jhu-cisst / mechatronics-software

mechatronics software
9 stars 9 forks source link

The Crash inside FirewirePort::WriteAllBoards #7

Closed alexunder closed 5 years ago

alexunder commented 5 years ago

I am using Xenomai patched ubuntu workstation. But when some Xenomai tasks ran with the 1k frequency, there would generate the memory exception, actually, these tasks wound invoke FirewirePort::WriteAllBoards. It seems that the real core-dump event happened inside libraw1394, but I think you guys may meet with this problem before, can you give me some hints to fix it, thanks very much. The core dump call trace is below:

#0  0x00007fd6db907428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007fd6db90902a in __GI_abort () at abort.c:89
#2  0x00007fd6db9497ea in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7fd6dba62ed8 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:175
#3  0x00007fd6db95237a in malloc_printerr (ar_ptr=<optimized out>, ptr=<optimized out>, str=0x7fd6dba62fa0 "double free or corruption (fasttop)", action=3) at malloc.c:5006
#4  _int_free (av=<optimized out>, p=<optimized out>, have_lock=0) at malloc.c:3867
#5  0x00007fd6db95653c in __GI___libc_free (mem=<optimized out>) at malloc.c:2968
#6  0x00007fd6dc2741c5 in ?? () from /usr/lib/x86_64-linux-gnu/libraw1394.so.11
#7  0x00007fd6dc274e22 in fw_loop_iterate () from /usr/lib/x86_64-linux-gnu/libraw1394.so.11
#8  0x00007fd6dc274f50 in ?? () from /usr/lib/x86_64-linux-gnu/libraw1394.so.11
#9  0x00007fd6dc2761cd in fw_read () from /usr/lib/x86_64-linux-gnu/libraw1394.so.11
#10 0x000000000054a9c1 in FirewirePort::ReadBlock(unsigned char, unsigned long, unsigned int*, unsigned int) ()
#11 0x0000000000549722 in FirewirePort::ReadAllBoards() ()
#12 0x0000000000523e79 in Console::readAll (this=0x1873860) at /home/psm/alexander/product/KayakController/Console/dVRK/Console.cpp:273
#13 0x00000000004cf788 in stateProc (arg=0x0) at /home/psm/alexander/product/KayakController/main.cpp:699
#14 0x00007fd6dd774baa in task_entry () from /usr/xenomai/lib/libalchemy.so.0
#15 0x00007fd6dd56115b in thread_trampoline () from /usr/xenomai/lib/libcopperplate.so.0
#16 0x00007fd6dd34990c in cobalt_thread_trampoline () from /usr/xenomai/lib/libcobalt.so.2
#17 0x00007fd6dcf1f6ba in start_thread (arg=0x7fd6dd9d4700) at pthread_create.c:333
#18 0x00007fd6db9d941d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
pkazanzides commented 5 years ago

Sorry, we have not tried to run the dVRK with Xenomai lately, mostly because you gain very little performance because the FireWire driver is not real-time. The last person who did FireWire testing on Xenomai was @zchen24, but that was quite some time ago.

zchen24 commented 5 years ago

@alexunder As Peter mentioned, the default Linux FireWire driver is not real-time. I did some testing a while ago and confirmed that running on Xenomai patched kernel still has overrun. If hard RT is crucial to you, the most promising route is to use the RT-FireWire (Zhang2005) driver. However, the driver was written for a really old kernel (2.6) and is no longer maintained. On the other hand, hard-RT is not necessary for the majority of applications including rendering haptics at multi kHz rate.

alexunder commented 5 years ago

@pkazanzides @zchen24 Thanks for your replies. I have done some work-arounds.

alexunder commented 5 years ago

So I should close it. Thanks!