search

ros2 / ros2_embedded_nuttx

This repository isn't actively being worked on. If you would like to take over maintainership please open a ticket on https://github.com/ros2/ros2
72 stars 34 forks source link

Packages sent by the embedded board corrupted #14

Closed vmayoral closed 9 years ago

vmayoral commented 9 years ago

It seems like the embedded boards sends bad locator information. Particularly:

PID_DEFAULT_LOCATOR_UNICAST
|_ Locator
              |_ kind

We expect 0x0...01 but get we 0x2000e201. The IP addresses of the locator also seem to have :: as a prefix which shouldn't happen.

vmayoral commented 9 years ago

Apparently the issue is not related to the code but to the memory (review https://github.com/brunodebus/tinq-core/issues/7).

It seems that the allocator is assigning addreses in the CCM memory sections which is unclear how it should work. In order to avoid this matter a flag in the NuttX kernel has been used (which disables this memory) however the code runs out of memory:

nsh> dds_chat
Network configured, starting DDS chat:
List pools created.
String pool initialized.
Typecode pools created.
Typesupport initialized.
Socket handler initialized.
Timer pool initialized.
Data buffer pools created.
Random host identifier generated.
Unique GUID prefix created: 6004b4df:00000004:00060000
Locator pools created.
History cache pools created.
Entities: reserved=98, maximum=4294967295
Domain pools created.
RTPS Initialised.
IP: scope = link..global
IP interfaces:
    eth0     : 192.168.0.3
RTPS over IPv4 Initialised.
QoS pools initialized.
DDS: core thread created.
str_new: out of memory!
Can't register ParticipantMessageData type!
Exiting program!
vmayoral commented 9 years ago

Assigning a stack size for each thread of 8192:

Network configured, starting DDS chat:
List pools created.
String pool initialized.
Typecode pools created.
Typesupport initialized.
Socket handler initialized.
Timer pool initialized.
Data buffer pools created.
Random host identifier generated.
Unique GUID prefix created: 600934df:00000004:00060000
Locator pools created.
History cache pools created.
Entities: reserved=98, maximum=4294967295
Domain pools created.
RTPS Initialised.
IP: up_hardfault: PANIC!!! Hard fault: 40000000
up_assert: Assertion failed at file:armv7-m/up_hardfault.c line: 183
up_dumpstate: sp:         20014a70
up_dumpstate: stack base: 20015120
up_dumpstate: stack size: 00001fe4
up_stackdump: 20014a60: 20014a60 20014a60 20014a70 20014d74 2000b350 00000000 20014a80 08001ccf
up_stackdump: 20014a80: 20014a70 20012cd0 00001fe4 20015120 20014aa0 08001d59 000000b7 20014ab0
up_stackdump: 20014aa0: 000000b7 080a7c38 20014ab0 08001e85 20014b00 00000003 20014ac0 68fb0001
up_stackdump: 20014ac0: 08005988 20014b00 20014ad0 08002e37 20014b00 00000003 00000000 08001e01
up_stackdump: 20014ae0: 20014ae8 08001583 20014b00 00000003 20012e7c 00000000 20014b48 0800047f
up_stackdump: 20014b00: 20014b48 00000000 20014d74 2000b350 00000000 20014b48 00000000 00000000
up_stackdump: 20014b20: 00000000 00000000 20012dc0 00000001 00000000 12e4f647 000007ff 08004c47
up_stackdump: 20014b40: 0800598a 21000000 0000000a 20012dc0 00000000 12e4f647 20014b60 20014b60
up_stackdump: 20014b60: 20014b68 20012da0 20012ea0 000016f0 20014b78 08004e77 20014b80 20012da0
up_stackdump: 20014b80: 20014b88 08004ef7 00000001 20012cd0 00003849 20012da0 20014ba0 0800454d
up_stackdump: 20014ba0: 20012e7c 00000000 00000001 20012cd0 20014bb8 08003515 20014bd0 00000001
up_stackdump: 20014bc0: 080b0dc0 20012cd0 20014bd0 08068ff3 20014c3c 080b0e28 00000082 04014bfc
up_stackdump: 20014be0: 000000ff 0004083c 00000000 080b1bc8 20014c3c 20014d74 20014c00 080a282d
up_stackdump: 20014c00: 20014d4c 080b48cc 00000100 20014c3c 080a2839 08006b55 00000072 20014c3c
up_stackdump: 20014c20: 000000ff 20014d74 20014c38 080697f5 00000001 0000000f 20014c40 61657263
up_stackdump: 20014c40: 625f6574 746c6975 655f6e69 6f70646e 3a746e69 6e616320 63207427 74616572
up_stackdump: 20014c60: 75622065 69746c69 6f74206e 20636970 44455328 69756250 6e69746c 73627553
up_stackdump: 20014c80: 70697263 6e6f6974 61655273 2f726564 73627553 70697263 6e6f6974 6c697542
up_stackdump: 20014ca0: 546e6974 6369706f 61746144 20002129 00004cb8 00000000 00000000 20014cc0
up_stackdump: 20014cc0: 20014cc8 00001021 00000000 20012cd0 20014cd8 080a0e01 2001ff98 20004d2c
up_stackdump: 20014ce0: 20014ce8 00000000 00000004 20014d74 20014cf8 08067d55 2001ff80 20004cf0
up_stackdump: 20014d00: 20014d08 08018483 2001ff80 2001ebb0 20014d20 080188ed 20014d20 08018491
up_stackdump: 20014d20: 080b1bac 080b1ae4 00000000 2001ebb0 0000000b ffffffff 20014d40 20014d4c
up_stackdump: 20014d40: 20014d68 0808efe1 080b48cc 080b1ae4 080b1bac 080b1ae4 00000000 00004d60
up_stackdump: 20014d60: 00000000 20012cd0 00000001 00000000 05000000 2001ebb0 00000000 00000000
up_stackdump: 20014d80: 2001f210 20014dac 20014da0 0808f433 00000000 20007858 2001ef10 39203d20
up_stackdump: 20014da0: 00000000 00000000 01014db8 2001ebb0 00000002 00000000 00000000 00000000
up_stackdump: 20014dc0: 00000000 00000001 ffffffff ffffffff ffffffff 7fffffff 7fffffff 00000000
up_stackdump: 20014de0: 00000000 00000000 7fffffff 7fffffff 00000000 00000000 05f5e100 00000000
up_stackdump: 20014e00: 00000000 00000001 00000001 7fffffff 7fffffff 00000000 00000000 00000000
up_stackdump: 20014e20: 7fffffff 7fffffff 00000001 00000000 05f5e100 00000000 00000000 00000001
up_stackdump: 20014e40: ffffffff ffffffff ffffffff 00000000 00000000 00000001 00000001 00000000
up_stackdump: 20014e60: 00000000 00000000 00000000 7fffffff 7fffffff 7fffffff 7fffffff 2001fce0
up_stackdump: 20014e80: 2001fe40 00000000 2001fce0 00000000 00000007 080b1a18 2001ffd8 2001ffa8
up_stackdump: 20014ea0: 2001ff80 080b1bac 00000000 2001fe40 7fffffff 2001ebe8 20014ee0 08091463
up_stackdump: 20014ec0: 00000001 00000000 00000001 00000000 2001f1e0 2001f240 00000000 2001f950
up_stackdump: 20014ee0: 0808ed79 2001ebb0 2001fce0 00000000 20014f18 080917b9 00000000 00000000
up_stackdump: 20014f00: 00000000 2001ef10 2001f1e0 2001f240 00000000 2000633c 2000b320 2001ebb0
up_stackdump: 20014f20: 00000001 20012cd0 20014f30 08068ba1 00000000 0000000d 2001f950 00000000
up_stackdump: 20014f40: 2001f240 2001f1e0 20014f50 08057e15 2001f240 2001ebb0 00000000 2001ecc0
up_stackdump: 20014f60: 2001f330 2001f260 20014f78 080724af 2001ebe8 2001ef1c 20014f80 2001ebb0
up_stackdump: 20014f80: 00000001 2001ebb0 20014f90 2001ef08 20014f98 080491c9 2000b470 2001ebb0
up_stackdump: 20014fa0: 0000b4f4 00000000 20014fb0 0804867b 00003849 00000000 20000ed0 00000000
up_stackdump: 20014fc0: 20015124 df340960 04005f00 00000600 2001ebb0 0000005f 00000000 00000000
up_stackdump: 20014fe0: 20014ff0 08011451 00000000 00000000 20015124 00000001 00ffffff 00000000
up_stackdump: 20015000: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
up_stackdump: 20015020: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
up_stackdump: 20015040: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
up_stackdump: 20015060: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
up_stackdump: 20015080: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
up_stackdump: 200150a0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
up_stackdump: 200150c0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
up_stackdump: 200150e0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
up_stackdump: 20015100: 20015108 080034b7 00000000 00000000 20012cd0 00000001 00000000 00000000
up_registerdump: R0: 20012dc0 00000001 00000000 12e4f647 20014d74 2000b350 00000000 20014b48
up_registerdump: R8: 00000000 00000000 00000000 00000000 000007ff 20014b48 08004c47 0800598a
up_registerdump: xPSR: 21000000 PRIMASK: 00000000 CONTROL: 00000000
vmayoral commented 9 years ago

It seems like:

CONFIG_IDLETHREAD_STACKSIZE=4096
CONFIG_USERMAIN_STACKSIZE=4096
CONFIG_PTHREAD_STACK_MIN=1024
CONFIG_PTHREAD_STACK_DEFAULT=4096

Is the minimum set of stack sizes for not getting hard faults

vmayoral commented 9 years ago

Modifying the MAX_TX_SIZE (and MAX_RX_SIZE) seems to play a role. Sizes of 8K make the program crash as soon as it starts while smaller ones delay the crash until the program starts sending packages to the network.

vmayoral commented 9 years ago 
CONFIG_IDLETHREAD_STACKSIZE=4096
CONFIG_USERMAIN_STACKSIZE=4096
CONFIG_PTHREAD_STACK_MIN=2048
CONFIG_PTHREAD_STACK_DEFAULT=4096

Seems to be a reasonable combination.

vmayoral commented 9 years ago

This seems to be an issue related to the ringbuffer implementation. Besided this matter, the system seems to be running out of memory and is allocating pointers in invalid regions.

vmayoral commented 9 years ago

Packages are sent properly. Tested reading byte a byte and compared with wireshark.

vmayoral commented 9 years ago

This was caused due to the fact that NuttX can't have a single socket sending and receiving simultaneously.