search

ApexAI / performance_test

**This project is deprecated** Go to https://gitlab.com/ApexAI/performance_test
64 stars 41 forks source link

Reliability issue? #70

Open LanderU opened 5 years ago

LanderU commented 5 years ago

Hello!

I'm not sure if this is actually a problem with my set of test or with the code itself.

Description:

We're running long term tests using your tool.

performance_test commit: https://github.com/ApexAI/performance_test/commit/04430edd8146ded8b82ea8ceb3026847c7351f26

ROS_DISTRO: Dashing https://github.com/ros2/ros2/tree/release-dashing-20190614

Performance_test command:

ros2 run performance_test perf_test -c ROS2 -l log -t Array1k

We're running this in loopback.

Test result:

After more or less 4-5h, is not deterministic, the output stop get stuck and not more output is printed. We need this data in order to plot the results.

In order to check if this is on the computer that is running the tests. I run it on a official dashing docker, and the result is the same, in other machine.

GDB output:

(gdb) where full
#0  0x00005580183e7818 in performance_test::DataRunner<performance_test::ROS2CallbackCommunicator<performance_test::topics::Array1k> >::sync_reset()

We need infinitive tests for the application that we're building, I'm not sure if there is a problem in my setup or not.

It would be great if you can advise me. I can provide you more information if needed.

Thanks in advance!

deeplearningrobotics commented 5 years ago

@LanderU: So the tool is stuck in the sync_reset function? Do you know which line? Maybe this is a race condition.

LanderU commented 5 years ago

So the tool is stuck in the sync_reset function?

You're right.

These are the candidates:

Thank you!

deeplearningrobotics commented 5 years ago

@LanderU: So it is stuck in the lock because another thread did not free it. Can you check for the other thread which is most likely stuck in the ROS 2 communicator class? The error you are seeing is just a symptom of this other thread being stuck.

dejanpan commented 5 years ago

@LanderU are you still having this problem or can we close this issue?

@esteve fyi

LanderU commented 5 years ago

The last time that I tested the problem still persists. I'll try it again.