Reduce excessive error and warning logs on Autoware launch

xmfcx commented 11 months ago

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and no duplicate issues were found.
[X] I've agreed with the maintainers that I can plan this task.

Description

When Autoware is executed, it generates a significant number of warning and error logs, despite the system running as expected. The logs are not only excessive but also misrepresent the stability of the running application, as they appear even in the absence of actual functional issues. This creates a noisy debugging environment, making it challenging to identify and address genuine issues.

The issue was replicated using the latest main branch with the planning simulator demo as per the Autoware documentation: Planning Simulation Tutorial.

A video demonstrating the issue is available here:

Autoware Error and Warning Logs Video

The logs from the run that showcase the problem can be found here:

Autoware Logs Gist

Purpose

The primary goal is to clean up the logs by either resolving the underlying causes of the warnings and errors or by adjusting the threat levels if the logs are not indicative of real issues. This will aid in de-cluttering the console output, making it easier to debug actual problems, and generally improve the developer experience.

Possible approaches

Log Analysis: Go through the logs to determine if the warnings/errors are justified. If they represent real issues, then they should be addressed with bug fixes.
Adjust Log Levels: If the logs are overly cautious and do not represent actual errors or critical warnings, then the log levels should be adjusted accordingly.
Code Refactoring: Inspect the sections of the code-base that are generating these logs and refactor if necessary to prevent false positives.
Configuration Tweaks: Check if the verbosity of the logs can be configured, allowing users to set the desired level of log detail.
Documentation Update: If certain logs are expected and do not indicate problems, this should be clearly documented to avoid confusion. (last resort, not preferred)
Incremental Fixing: Tackle the errors and warnings in batches, creating multiple issues or pull requests to address them systematically.

Definition of done

[ ] The error and warning logs that are not indicative of actual issues are eliminated or their threat levels are adjusted.
[ ] The logging output should be clear and should accurately reflect the operational status of Autoware.
[ ] Documentation should be updated, if necessary, to reflect any changes made to log management.
[ ] Additional issues should be created for any unresolved logs that require further investigation or long-term changes.
[ ] The Autoware community agrees that the logging is now informative and not excessively verbose, allowing for effective debugging.

xmfcx commented 11 months ago

To clarify, this task encapsulates warning and error logs coming from any part of the Autoware for any reason.

ahmeddesokyebrahim commented 9 months ago

I would like to share updates regarding this issue.

The target of this issue is to reduce excessive log message in Autoware for the following tutorials with default parameters

In this sheet, we are categorize and following up the log messages during the fixing process of this issue.

The initial plan is to have the task into 5 phases :

Phase	Definition
Phase 1	Planning simulation : starting planning simulator and before setting vehicle start location. This phase produces a lot of recurrent message while nodes are initializing
Phase 2	Planning simulation : setting vehicle start location, goal location, and basic driving scenario. This phase where actually the nodes are starting to work and communicate.
Phase 3	Planning simulation : handing special driving scenarios in planning simulator like parking, …
Phase 4	Rosbag replay simulation
Phase 5	AWSIM (extended goal)

The following is the list of relevant PRs for proposing solution fixing the different log messages. For completing this issue, these PRs must be successully reviewed and merged [list can be updated during development]

ahmeddesokyebrahim commented 8 months ago

This is the list of bugs opened for asking the community to collaborate in fixing some of the logs messages and their associated PRs:

Phase 1, 2, and 3

Phase 4

Bug	PR
https://github.com/autowarefoundation/autoware.universe/issues/6204	https://github.com/autowarefoundation/autoware.universe/pull/6405
https://github.com/autowarefoundation/autoware.universe/issues/6205	https://github.com/autowarefoundation/autoware.universe/pull/6298	https://github.com/autowarefoundation/autoware.universe/issues/6379	--
https://github.com/autowarefoundation/autoware.universe/issues/6378	https://github.com/autowarefoundation/autoware.universe/pull/6696
https://github.com/autowarefoundation/autoware.universe/issues/6380	https://github.com/autowarefoundation/autoware.universe/pull/6696
https://github.com/autowarefoundation/autoware.universe/issues/6381	https://github.com/autowarefoundation/autoware.universe/pull/6696
https://github.com/autowarefoundation/autoware.universe/issues/6379	https://github.com/autowarefoundation/autoware.universe/pull/6713
https://github.com/tier4/nebula/issues/122	https://github.com/tier4/nebula/pull/118
https://github.com/autowarefoundation/autoware.universe/issues/6383	--
https://github.com/autowarefoundation/autoware.universe/issues/6515	--

ahmeddesokyebrahim commented 8 months ago

The PRs for the first 3 phases are ready for review. Recommended Merge Sequence

The following one does not matter in sequence

https://github.com/tier4/pointcloud_to_laserscan/pull/6 [does not matter]

The following phases will be addressed in separate PRs.

ahmeddesokyebrahim commented 7 months ago

List of PR associated to this issue

Merged :

Ready for Review

to be updated if any

WIP

to be updated if any

stale[bot] commented 2 months ago

This pull request has been automatically marked as stale because it has not had recent activity.

autowarefoundation / autoware.universe