search

luxonis / depthai-core

DepthAI C++ Library
MIT License
238 stars 128 forks source link

Sporadic crash on RPI - Segmentation fault on latest depthai #567

Open Fguer12 opened 2 years ago

Fguer12 commented 2 years ago

Hello we got the following error message after running the pipeline for around 8 hours.

Error on 24.8: "Bus Error (Invalid Address Alignment [0xf19)" Error on 25.8: ""Segmentation fault /Address not mapped to obeject [0x10])" cam3_25 8

Both errors have been added as screenshots above.

the MRE file is attached as well below: MRE_depthai.py.zip

themarpe commented 2 years ago

Thanks for the report @Fguer12 and sorry for the issues. We'll be checking this out in more detail this week and will get back to you as soon as we have more information on it.

themarpe commented 2 years ago

@Fguer12 are you using the latest depthai library (2.17.3.1)?

Fguer12 commented 2 years ago

Hello thanks for the reply. We were running depthai-2.15.1.0. I upgraded to the latest version. Lets see if this helps

Fguer12 commented 2 years ago

Hey @themarpe we have installed the latest library now and it did not crash between the 25th and 1.09. However, today it crashed again with the same error message. Any advice how to handle this?

themarpe commented 2 years ago

@Fguer12 Thanks for the feedback on this. One extra thing you may try is installing 2.17.1 instead. It has a reverted temporary fix for some stability issues, but it made some issues in other cases, which we are fixing.

Otherwise, did the issue present itself as an segfault or just an exception thrown for communication error? If the latter, try catching that and upon this, reconnect. if the former, we are investigating some XLink issues which will should address the segfault scenarios upon conn/disconnect events

Fguer12 commented 2 years ago

Hi there

the issue is presenting itself as a segfault, so catching it does not work.

Regarding the versioning. We reverted to 2.17.1 and the same issue happens but not after approx 10 hours but after approx 18 hours. We now went back to the latest depthai version and the issue still persists after approx 7-8 hours so it has become a bit worse. Will you prioritize the segfault issues in your next sprints/OKRs/PIs?

themarpe commented 2 years ago

@Fguer12 do you mind testing the following branch: xlink_race_improvements It's a "Work-in-progress" that aims to address these segfault issues.

Let me know how that works for you