system_error_monitor is reporting diag_array msg timeout during rosbag replay simulation initialization

ahmeddesokyebrahim commented 9 months ago

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and no duplicate issues were found.
[X] I'm convinced that this is not my fault but a bug.

Description

Part of:

https://github.com/autowarefoundation/autoware.universe/issues/5539

Currently, we are working on a task for fixing and improving Autoware logging system as Autoware creates a noisy debugging environment, making it challenging to identify and address genuine issues.

For more details about the issue, plan how to tackle it, and linked PR, please have a look here.

For this bug, when starting rosbag replay simulation and before replaying the rosbag, system_error_monitor is reporting diag_array msg timeout as follow :

[system_error_monitor-22] [ERROR 1708457713.178562834] [system.system_error_monitor]: diag_array msg is timeout... (isDataHeartbeatTimeout() at ~/autoware/src/universe/autoware.universe/system/system_error_monitor/src/system_error_monitor_core.cpp:420)

Expected behavior

system_error_monitor is reporting diag_array msg timeout when data is actually sent then timeout occurs or after countdown during initialization

Actual behavior

system_error_monitor is directly reporting diag_array msg timeout during initialization

Steps to reproduce

Start rosbag replay simulation
Before replaying the sample rosbag, you can see the aforementioned error message.

Versions

No response

Possible causes

No response

Additional context

In the context of a similar issue, @xmfcx proposed the following : In real vehicle, this error message could be expected to be seen even when the topic is never received.

I have 2 proposals to address these:

1. System Error Monitor Init Phase

Allow system_error_monitor node to have an initial countdown (user can set the seconds) and It won't report anything for this period. After it ends, it will publish as usual.

This period can be set to something like 10s for planning simulator maybe.

2. Autoware RViz Side Panel Visualization

We could have this somehow shown as red in the future side panel for this topic. And once this error is gone, it would turn green for this message.

(This panel issue still hasn't been created so we can skip this solution for now.)

Here is the full comment and context I am copying the proposal from for reference

ahmeddesokyebrahim commented 9 months ago

@ito-san : I would highly appreciate if you can have a look and support in this issue or let us know who is the current package owner for system_error_monitor that can support here.

stale[bot] commented 7 months ago

This pull request has been automatically marked as stale because it has not had recent activity.

meliketanrikulu commented 6 months ago

Hello @ahmeddesokyebrahim are you working on this issue. If so, can you assign it to yourself?

ahmeddesokyebrahim commented 6 months ago

Hello @ahmeddesokyebrahim are you working on this issue. If so, can you assign it to yourself?

Hi @meliketanrikulu This issue was created as part of the efforts within reducing Autoware excessive logs task. All the created issues were closed except those may require further investigation or long-term changes. You can have a look to the addition context in the issue description above to understand what I mean. BTW, the same applies for this issue as well. So, yes I have opened these issues, but actually I am not working on them and that is why they are not assigned to me.

meliketanrikulu commented 6 months ago

So, yes I have opened these issues, but actually I am not working on them and that is why they are not assigned to me.

@ahmeddesokyebrahim Just wanted to check if there is an update to the assignee section. I thought it might be part of your PR. Thanks for the information.

stale[bot] commented 4 months ago

This pull request has been automatically marked as stale because it has not had recent activity.

autowarefoundation / autoware.universe