search

cntools / libsurvive

Open Source Lighthouse Tracking System
MIT License
722 stars 132 forks source link

Assertion failed: ctx->poseLength < 20 #281

Open bl4ckb0ne opened 1 year ago

bl4ckb0ne commented 1 year ago

I'm hitting the assertion here in src/survive_optimizer.c after a few minutes of run (varying from 2 mins to 15 mins)

Here's the log from survive-cli --force-calibrate after removing the config file in ~/.config/libsurvive/config.json. Both controllers (Valve Knuckles) are lying flat on my desk, untouched, and the HMD (Valve index) on my head.

$ survive-cli --force-calibrate
Info: Loaded drivers: GlobalSceneSolver, HTCVive
Info: Force calibrate flag set -- clearing position on all lighthouses
Info: Adding tracked object KN0 from HTC
Info: Adding tracked object T20 from HTC
Info: Adding tracked object KN1 from HTC
Info: Device KN0 has watchman FW version 1546408689 and FPGA version 538/10/2; named '                       watchman'. Hardware id 0x110e0009 Board rev: 3 (len 56)
Info: Device KN1 has watchman FW version 1546408689 and FPGA version 538/10/2; named '                       watchman'. Hardware id 0x110e0109 Board rev: 3 (len 56)
Info: Device T20 has watchman FW version 1555018800 and FPGA version 538/9/2; named '                       watchman'. Hardware id 0x22154109 Board rev: 3 (len 56)
Info: T20 is treated as HMD device
Info: Detected LH gen 2 system.
Info: Adding lighthouse ch 1 (idx: 0, cnt: 1)
Info: OOTX not set for LH in channel 1; attaching ootx decoder using device T20
Info: LightcapMode (KN0) 1 -> 2 (ff)
Info: LightcapMode (KN1) 1 -> 2 (ff)
Info: LightcapMode (T20) 1 -> 2 (4)
Info: Adding lighthouse ch 0 (idx: 1, cnt: 2)
Info: OOTX not set for LH in channel 0; attaching ootx decoder using device T20
Info: (1) Preamble found
Info: (0) Preamble found
Info: Got OOTX packet 1 f0ade8db
Info: Got OOTX packet 0 d6ab3660
Info: MPFIT success 3896455.778023/759.5648898653/0.0001765 (132 measurements, 1, MP_OK_CHI, 13 iters, up err 0.0003941, trace 0.0005577)
Info: Global solve with 3 scenes for 0 with error of 3896455.778023/759.5648898653 (acc err 0.0003)
Info: Global solve with 3 scenes for 1 with error of 3896455.778023/759.5648898653 (acc err 0.0010)
Info: Using LH 1 (d6ab3660) as reference lighthouse
Info: MPFIT success 6388.816801/999.8663550295/0.0001847 (190 measurements, 1, MP_OK_CHI, 5 iters, up err 0.0003125, trace 0.0000066)
Info: Global solve with 4 scenes for 0 with error of 6388.816801/999.8663550295 (acc err 0.0003)
Info: Global solve with 4 scenes for 1 with error of 6388.816801/999.8663550295 (acc err 0.0010)
Info: Using LH 1 (d6ab3660) as reference lighthouse
Info: MPFIT success 6490.324541/1169.5468964587/0.0001797 (246 measurements, 1, MP_OK_CHI, 4 iters, up err 0.0002505, trace 0.0000084)
Info: Global solve with 5 scenes for 0 with error of 6490.324541/1169.5468964587 (acc err 0.0002)
Info: Global solve with 5 scenes for 1 with error of 6490.324541/1169.5468964587 (acc err 0.0010)
Info: Using LH 1 (d6ab3660) as reference lighthouse
Info: MPFIT success 6159.686096/1398.7625659923/0.0001815 (305 measurements, 4, MP_OK_DIR, 5 iters, up err 0.0002203, trace 0.0000100)
Info: Global solve with 6 scenes for 0 with error of 6159.686096/1398.7625659923 (acc err 0.0002)
Info: Global solve with 6 scenes for 1 with error of 6159.686096/1398.7625659923 (acc err 0.0011)
Info: Using LH 1 (d6ab3660) as reference lighthouse
Info: MPFIT success 30100.113315/2275.6997423862/0.0002300 (358 measurements, 1, MP_OK_CHI, 4 iters, up err 0.0001954, trace 0.0000166)
Info: Global solve with 7 scenes for 0 with error of 30100.113315/2275.6997423862 (acc err 0.0002)
Info: Global solve with 7 scenes for 1 with error of 30100.113315/2275.6997423862 (acc err 0.0010)
Info: Using LH 1 (d6ab3660) as reference lighthouse
Info: MPFIT success 15883.534761/2802.2547521809/0.0002389 (414 measurements, 1, MP_OK_CHI, 4 iters, up err 0.0001831, trace 0.0000200)
Info: Global solve with 8 scenes for 0 with error of 15883.534761/2802.2547521809 (acc err 0.0002)
Info: Global solve with 8 scenes for 1 with error of 15883.534761/2802.2547521809 (acc err 0.0011)
Info: Using LH 1 (d6ab3660) as reference lighthouse
Info: MPFIT success 77581.906085/3437.6292149815/0.0002384 (485 measurements, 1, MP_OK_CHI, 5 iters, up err 0.0001928, trace 0.0000200)
Info: Global solve with 10 scenes for 0 with error of 77581.906085/3437.6292149815 (acc err 0.0002)
Info: Global solve with 10 scenes for 1 with error of 77581.906085/3437.6292149815 (acc err 0.0011)
Info: Using LH 1 (d6ab3660) as reference lighthouse
Info: MPFIT success 94036.512484/3590.2466779616/0.0002315 (541 measurements, 1, MP_OK_CHI, 4 iters, up err 0.0001763, trace 0.0000204)
Info: Global solve with 11 scenes for 0 with error of 94036.512484/3590.2466779616 (acc err 0.0002)
Info: Global solve with 11 scenes for 1 with error of 94036.512484/3590.2466779616 (acc err 0.0011)
Info: Using LH 1 (d6ab3660) as reference lighthouse
Info: MPFIT success 109782.623070/4278.9952810125/0.0002325 (594 measurements, 2, MP_OK_PAR, 6 iters, up err 0.0002124, trace 0.0000131)
Info: Global solve with 13 scenes for 0 with error of 109782.623070/4278.9952810125 (acc err 0.0003)
Info: Global solve with 13 scenes for 1 with error of 109782.623070/4278.9952810125 (acc err 0.0010)
Info: Using LH 1 (d6ab3660) as reference lighthouse
Info: MPFIT success 59512.301112/4515.6896849411/0.0002309 (619 measurements, 1, MP_OK_CHI, 4 iters, up err 0.0002256, trace 0.0000131)
Info: Global solve with 14 scenes for 0 with error of 59512.301112/4515.6896849411 (acc err 0.0003)
Info: Global solve with 14 scenes for 1 with error of 59512.301112/4515.6896849411 (acc err 0.0009)
Info: Using LH 1 (d6ab3660) as reference lighthouse
Info: MPFIT success 70188.150794/4796.4519054181/0.0002304 (649 measurements, 3, MP_OK_BOTH, 5 iters, up err 0.0002262, trace 0.0000115)
Info: Global solve with 15 scenes for 0 with error of 70188.150794/4796.4519054181 (acc err 0.0003)
Info: Global solve with 15 scenes for 1 with error of 70188.150794/4796.4519054181 (acc err 0.0009)
Info: Using LH 1 (d6ab3660) as reference lighthouse
Info: MPFIT success 109645.252213/5513.4051769925/0.0002490 (661 measurements, 1, MP_OK_CHI, 5 iters, up err 0.0002291, trace 0.0000127)
Info: Global solve with 16 scenes for 0 with error of 109645.252213/5513.4051769925 (acc err 0.0003)
Info: Global solve with 16 scenes for 1 with error of 109645.252213/5513.4051769925 (acc err 0.0009)
Info: Using LH 1 (d6ab3660) as reference lighthouse
Info: MPFIT success 132169.909869/5849.7465375743/0.0002501 (691 measurements, 1, MP_OK_CHI, 5 iters, up err 0.0002248, trace 0.0000115)
Info: Global solve with 17 scenes for 0 with error of 132169.909869/5849.7465375743 (acc err 0.0004)
Info: Global solve with 17 scenes for 1 with error of 132169.909869/5849.7465375743 (acc err 0.0009)
Info: Using LH 1 (d6ab3660) as reference lighthouse
Info: MPFIT success 112793.956355/6908.3823041417/0.0002585 (743 measurements, 1, MP_OK_CHI, 5 iters, up err 0.0002422, trace 0.0000116)
Info: Global solve with 19 scenes for 0 with error of 112793.956355/6908.3823041417 (acc err 0.0004)
Info: Global solve with 19 scenes for 1 with error of 112793.956355/6908.3823041417 (acc err 0.0008)
Info: Using LH 1 (d6ab3660) as reference lighthouse
Assertion failed: ctx->poseLength < 20 (/home/simon/src/libsurvive/src/survive_optimizer.c: survive_optimizer_get_max_parameters_count: 338)

libsurvive has been build from source from 5319743334d04cef0f0dde5a2c07372006023952 with cnkalman 6b350314225e28d2e4e8daad7d2971d22386f76f and cnmatrix 5936c62511305227fbd59b2d5a43aaf89ec3a0b6

imcnanie commented 1 year ago

I'm seeing this error as well when using the https://github.com/asymingt/libsurvive_ros2 ros driver

bl4ckb0ne commented 1 year ago

FWIW running in release shuts down the assert and works fairly fine.

asymingt commented 1 year ago

I've been running the node in the background this morning. Not the same error, but here's a backtrace from one assert that was hit at runtime, admittedly in DebWithRelInfo mode. 

[INFO] [1689791531.441246374] [libsurvive_ros2]: Start listening for events..
[INFO] [1689791531.441258978] [libsurvive_ros2]: Cleaning up.
[New Thread 0x7fffe205b640 (LWP 3316)]
Info: Adding tracked object T20 from HTC
[INFO] [1689791531.465595271] [libsurvive_ros2]: A new device  was added at time 1689791531.466556
Info: Device T20 has watchman FW version 1592875850 and FPGA version 538/7/2; named '                       watchman'. Hardware id 0x84020109 Board rev: 3 (len 56)
Info: Detected LH gen 2 system.
Info: Adding lighthouse ch 0 (idx: 0, cnt: 1)
Info: OOTX not set for LH in channel 0; attaching ootx decoder using device T20
Info: LightcapMode (T20) 1 -> 2 (4)
Info: Adding lighthouse ch 1 (idx: 1, cnt: 2)
Info: OOTX not set for LH in channel 1; attaching ootx decoder using device T20
Info: Adding lighthouse ch 2 (idx: 2, cnt: 3)
Info: OOTX not set for LH in channel 2; attaching ootx decoder using device T20
Info: Adding lighthouse ch 3 (idx: 3, cnt: 4)
Info: OOTX not set for LH in channel 3; attaching ootx decoder using device T20
Info: (1) Preamble found
Info: (0) Preamble found
Info: (2) Preamble found
Info: (3) Preamble found
Info: Got OOTX packet 1 75aad68b
Info: Got OOTX packet 0 68456b3d
Info: Got OOTX packet 2 81f01d94
Info: Got OOTX packet 3 d60829f8
Info: MPFIT success 1404725.316952/21624.2538415510/0.0009434 (251 measurements, 1, MP_OK_CHI, 28 iters, up err 0.0015864, trace 0.0020379)
Info: Global solve with 4 scenes for 0 with error of 1404725.316952/21624.2538415510 (acc err 0.0018)
Info: Global solve with 4 scenes for 1 with error of 1404725.316952/21624.2538415510 (acc err 0.0004)
Info: Global solve with 4 scenes for 2 with error of 1404725.316952/21624.2538415510 (acc err 0.0006)
Info: Global solve with 4 scenes for 3 with error of 1404725.316952/21624.2538415510 (acc err 0.0013)
Info: Using LH 0 (68456b3d) as reference lighthouse
[INFO] [1689791550.724349254] [libsurvive_ros2]: A new device LHB-68456B3D was added at time 1689791550.725333
[INFO] [1689791550.809408819] [libsurvive_ros2]: A new device LHB-75AAD68B was added at time 1689791550.725362
[INFO] [1689791550.812691672] [libsurvive_ros2]: A new device LHB-81F01D94 was added at time 1689791550.725387
[INFO] [1689791550.819145073] [libsurvive_ros2]: A new device LHB-D60829F8 was added at time 1689791550.725412
Info: MPFIT success 28278.936566/28160.5907828934/0.0009595 (316 measurements, 1, MP_OK_CHI, 5 iters, up err 0.0012136, trace 0.0006379)
Info: Global solve with 5 scenes for 0 with error of 28278.936566/28160.5907828934 (acc err 0.0018)
Info: Global solve with 5 scenes for 1 with error of 28278.936566/28160.5907828934 (acc err 0.0003)
Info: Global solve with 5 scenes for 2 with error of 28278.936566/28160.5907828934 (acc err 0.0006)
Info: Global solve with 5 scenes for 3 with error of 28278.936566/28160.5907828934 (acc err 0.0010)
Info: Using LH 0 (68456b3d) as reference lighthouse
libsurvive_ros2_node: /home/ubuntu/ros2_ws/build/libsurvive_ros2/libsurvive-prefix/src/libsurvive/redist/variance.h:18: variance_measure_add: Assertion `isfinite(d[i])' failed.

Thread 13 "libsurvive_ros2" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffe3fff640 (LWP 3312)]
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140737018590784) at ./nptl/pthread_kill.c:44
44      ./nptl/pthread_kill.c: No such file or directory.
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737018590784) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140737018590784) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140737018590784, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007ffff7922476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff79087f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x00007ffff790871b in __assert_fail_base (fmt=0x7ffff7abd150 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", 
    assertion=0x7ffff0b48186 "isfinite(d[i])", 
    file=0x7ffff0b48128 "/home/ubuntu/ros2_ws/build/libsurvive_ros2/libsurvive-prefix/src/libsurvive/redist/variance.h", 
    line=18, function=<optimized out>) at ./assert/assert.c:92
#6  0x00007ffff7919e96 in __GI___assert_fail (assertion=0x7ffff0b48186 "isfinite(d[i])", 
    file=0x7ffff0b48128 "/home/ubuntu/ros2_ws/build/libsurvive_ros2/libsurvive-prefix/src/libsurvive/redist/variance.h", 
    line=18, function=0x7ffff0b491c0 <__PRETTY_FUNCTION__.5> "variance_measure_add") at ./assert/assert.c:101
#7  0x00007ffff0b3a29a in variance_measure_add ()
   from /home/ubuntu/ros2_ws/build/libsurvive_ros2/libsurvive-install/lib/libsurvive/plugins/poser_mpfit.so
#8  0x00007ffff0b42a45 in solve_global_scene ()
   from /home/ubuntu/ros2_ws/build/libsurvive_ros2/libsurvive-install/lib/libsurvive/plugins/poser_mpfit.so
#9  0x00007ffff0b46b2c in PoserMPFIT ()
   from /home/ubuntu/ros2_ws/build/libsurvive_ros2/libsurvive-install/lib/libsurvive/plugins/poser_mpfit.so
#10 0x00007ffff767be25 in survive_threaded_poser_fn ()
   from /home/ubuntu/ros2_ws/install/libsurvive_ros2/lib/libsurvive.so.0
#11 0x00007ffff0b4fb20 in run_optimization ()
   from /home/ubuntu/ros2_ws/build/libsurvive_ros2/libsurvive-install/lib/libsurvive/plugins/driver_global_scene_solver.so
#12 0x00007ffff0b5124c in survive_threaded_gss_thread_fn ()
   from /home/ubuntu/ros2_ws/build/libsurvive_ros2/libsurvive-install/lib/libsurvive/plugins/driver_global_scene_solver.so
#13 0x00007ffff7974b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#14 0x00007ffff7a06a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
asymingt commented 1 year ago

FWIW running in release shuts down the assert and works fairly fine.

@bl4ckb0ne I should be able to relatively easily compile in Release mode, but I'm worried that these asserts signal some bigger issue that might be masked if we disable them.