zephyrproject-rtos / zephyr

Primary Git Repository for the Zephyr Project. Zephyr is a new generation, scalable, optimized, secure RTOS for multiple hardware architectures.
https://docs.zephyrproject.org
Apache License 2.0
10.43k stars 6.39k forks source link

Logging stops on all backend when overloaded for a time #75736

Closed thales-nascimento closed 1 month ago

thales-nascimento commented 1 month ago

Describe the bug Logging stops on all backend when overloaded for a time. After stopped no more messages are produced, even when the overload ceases. By overloaded I mean to drop 99% of the messages.

Platform: STM32H743 Version: tag zephyr-v3.5.0

Additional info: I have two backends, one logs on UART and other logs to an external flash with filesystem littlefs and file rotation. All my threas are same priority 5.

Workarround: patching log_core.c to trigger the logging thread when above the configured threshold seems to work. With the workarround the system behaves normally after the overload passes.

KConfigs ```txt # # Logging # CONFIG_LOG=y CONFIG_LOG_CORE_INIT_PRIORITY=0 CONFIG_LOG_MODE_DEFERRED=y # CONFIG_LOG_MODE_IMMEDIATE is not set # CONFIG_LOG_MODE_MINIMAL is not set # CONFIG_LOG_FRONTEND is not set # CONFIG_LOG_CUSTOM_HEADER is not set # CONFIG_LOG_MULTIDOMAIN is not set # # Logging levels filtering # # CONFIG_LOG_RUNTIME_FILTERING is not set CONFIG_LOG_DEFAULT_LEVEL=3 CONFIG_LOG_OVERRIDE_LEVEL=0 CONFIG_LOG_MAX_LEVEL=4 # end of Logging levels filtering # # Processing # CONFIG_LOG_PRINTK=y CONFIG_LOG_MODE_OVERFLOW=y # CONFIG_LOG_BLOCK_IN_THREAD is not set CONFIG_LOG_PROCESS_TRIGGER_THRESHOLD=10 CONFIG_LOG_PROCESS_THREAD=y CONFIG_LOG_PROCESS_THREAD_STARTUP_DELAY_MS=1000 CONFIG_LOG_PROCESS_THREAD_SLEEP_MS=1000 CONFIG_LOG_PROCESS_THREAD_STACK_SIZE=4096 # CONFIG_LOG_PROCESS_THREAD_CUSTOM_PRIORITY is not set CONFIG_LOG_BUFFER_SIZE=8192 CONFIG_LOG_TRACE_SHORT_TIMESTAMP=y # CONFIG_LOG_TIMESTAMP_64BIT is not set # CONFIG_LOG_SPEED is not set # end of Processing # # Output Formatting # # # Prepend non-hexdump log message with function name # # CONFIG_LOG_FUNC_NAME_PREFIX_ERR is not set # CONFIG_LOG_FUNC_NAME_PREFIX_WRN is not set # CONFIG_LOG_FUNC_NAME_PREFIX_INF is not set # CONFIG_LOG_FUNC_NAME_PREFIX_DBG is not set # end of Prepend non-hexdump log message with function name # CONFIG_LOG_MIPI_SYST_ENABLE is not set # CONFIG_LOG_THREAD_ID_PREFIX is not set # CONFIG_LOG_CUSTOM_FORMAT_SUPPORT is not set CONFIG_LOG_BACKEND_SHOW_COLOR=y # CONFIG_LOG_INFO_COLOR_GREEN is not set CONFIG_LOG_TAG_MAX_LEN=0 CONFIG_LOG_BACKEND_FORMAT_TIMESTAMP=y # CONFIG_LOG_OUTPUT_FORMAT_LINUX_TIMESTAMP is not set # CONFIG_LOG_OUTPUT_FORMAT_CUSTOM_TIMESTAMP is not set # end of Output Formatting # # Backends # # CONFIG_LOG_BACKEND_FS is not set # CONFIG_LOG_BACKEND_NET is not set # CONFIG_LOG_BACKEND_SWO is not set CONFIG_LOG_BACKEND_UART=y # CONFIG_LOG_BACKEND_UART_ASYNC is not set CONFIG_LOG_BACKEND_UART_BUFFER_SIZE=1 CONFIG_LOG_BACKEND_UART_AUTOSTART=y CONFIG_LOG_BACKEND_UART_OUTPUT_TEXT=y # CONFIG_LOG_BACKEND_UART_OUTPUT_DICTIONARY is not set # CONFIG_LOG_BACKEND_UART_OUTPUT_CUSTOM is not set CONFIG_LOG_BACKEND_UART_OUTPUT_DEFAULT=0 # CONFIG_LOG_BACKEND_IPC_SERVICE is not set # end of Backends # # Misc # CONFIG_LOG_DOMAIN_ID=0 CONFIG_LOG_USE_VLA=y # CONFIG_LOG_ALWAYS_RUNTIME is not set # CONFIG_LOG_FMT_SECTION is not set # CONFIG_LOG_USE_TAGGED_ARGUMENTS is not set # CONFIG_LOG_MEM_UTILIZATION is not set CONFIG_LOG_FAILURE_REPORT_PERIOD=1000 # end of Misc CONFIG_LOG_OUTPUT=y # end of Logging ```

Expected behavior Messages should be dropped as necessary but the logging system should remain operational.

Impact minor, patch works for me.

Environment

github-actions[bot] commented 1 month ago

Hi @thales-nascimento! We appreciate you submitting your first issue for our open-source project. 🌟

Even though I'm a bot, I can assure you that the whole community is genuinely grateful for your time and effort. 🤖💙

pdgendt commented 1 month ago

Can you submit your fix as a PR?