espressif / arduino-esp32

Arduino core for the ESP32
GNU Lesser General Public License v2.1
13.73k stars 7.43k forks source link

fix(asyncudp): Fixes and implements tcpip thread locking #10415

Closed HamzaHajeir closed 1 month ago

HamzaHajeir commented 1 month ago

Description of Change

Implements and fixes TCPIP Core locking in AsyncUDP when enabled.

AsyncUDP calls Raw LwIP APIs that are not safe to call from other threads without locking. While some raw APIs are being called by using tcpip_api_call(), other calls are not considering this method, as [1], [2], [3], and [4].

When CONFIG_LWIP_CHECK_THREAD_SAFETY is enabled, the system gets aborted when the lwip core is not locked; Therefore exposing a source of bugs.

Any call to tcpip_api_call() in tcpip core locked results in a deadlock, thus removed the corresponding calls.

I've prefered to check against the state before locking and unlocking, helpful when any concerned AsyncUDP API being called from within LwIP thread, highly through callbacks, if any.

This will not affect the direct Arduino users but until esp32-arduino-libs-builder PR gets merged it will fix a source of bugs.

Currently: Arduino-as-ESPIDF-component users can easily have this fix by enabling CONFIG_LWIP_TCPIP_CORE_LOCKING option.

Related links

https://www.nongnu.org/lwip/2_1_x/multithreading.html

https://github.com/espressif/arduino-esp32/issues/10391#issuecomment-2385443646

CLAassistant commented 1 month ago

CLA assistant check
All committers have signed the CLA.

github-actions[bot] commented 1 month ago
Warnings
:warning: **Some issues found for the commit messages in this PR:** - the commit message `"fix(asyncudp): Adds missing unlock"`: - *summary* looks too short *** **Please fix these commit messages** - here are some basic tips: - follow [Conventional Commits style](https://www.conventionalcommits.org/en/v1.0.0/) - correct format of commit message should be: `(): `, for example `fix(esp32): Fixed startup timeout issue` - allowed types are: `change,ci,docs,feat,fix,refactor,remove,revert,test` - sufficiently descriptive message summary should be between 20 to 72 characters and start with upper case letter - avoid Jira references in commit messages (unavailable/irrelevant for our customers) `TIP:` Install pre-commit hooks and run this check when committing (uses the [Conventional Precommit Linter](https://github.com/espressif/conventional-precommit-linter)).
Messages
:book: You might consider squashing your 3 commits (simplifying branch history).

πŸ‘‹ Hello HamzaHajeir, we appreciate your contribution to this project!


Click to see more instructions ...


This automated output is generated by the PR linter DangerJS, which checks if your Pull Request meets the project's requirements and helps you fix potential issues.

DangerJS is triggered with each push event to a Pull Request and modify the contents of this comment.

Please consider the following:
- Danger mainly focuses on the PR structure and formatting and can't understand the meaning behind your code or changes.
- Danger is not a substitute for human code reviews; it's still important to request a code review from your colleagues.
- Resolve all warnings (⚠️ ) before requesting a review from human reviewers - they will appreciate it.
- Addressing info messages (πŸ“–) is strongly recommended; they're less critical but valuable.
- To manually retry these Danger checks, please navigate to the Actions tab and re-run last Danger workflow.

Review and merge process you can expect ...


We do welcome contributions in the form of bug reports, feature requests and pull requests.

1. An internal issue has been created for the PR, we assign it to the relevant engineer.
2. They review the PR and either approve it or ask you for changes or clarifications.
3. Once the GitHub PR is approved we do the final review, collect approvals from core owners and make sure all the automated tests are passing.
- At this point we may do some adjustments to the proposed change, or extend it by adding tests or documentation.
4. If the change is approved and passes the tests it is merged into the default branch.

Generated by :no_entry_sign: dangerJS against 848b0aeaf3b41c1baff1a5cd7e9696b62e8d0748

github-actions[bot] commented 1 month ago

Memory usage test (comparing PR against master branch)

The table below shows the summary of memory usage change (decrease - increase) in bytes and percentage for each target.

MemoryFLASH [bytes]FLASH [%]RAM [bytes]RAM [%]
TargetDECINCDECINCDECINCDECINC
ESP32S3000.000.00000.000.00
ESP32S2000.000.00000.000.00
ESP32C3000.000.00000.000.00
ESP32C6000.000.00000.000.00
ESP32000.000.00000.000.00
Click to expand the detailed deltas report [usage change in BYTES]
TargetESP32S3ESP32S2ESP32C3ESP32C6ESP32
ExampleFLASHRAMFLASHRAMFLASHRAMFLASHRAMFLASHRAM
AsyncUDP/examples/AsyncUDPClient0000000000
AsyncUDP/examples/AsyncUDPMulticastServer0000000000
AsyncUDP/examples/AsyncUDPServer0000000000
github-actions[bot] commented 1 month ago

Test Results

 56 files   -β€Š59   56 suites   -β€Š59   4m 12s :stopwatch: - 30m 21s  21 tests  -β€Šβ€‡7   21 :white_check_mark: + 4  0 :zzz: Β±0  0 :x: Β±0  135 runsβ€Š  -β€Š72  135 :white_check_mark:  -β€Š16  0 :zzz: Β±0  0 :x: Β±0 

Results for commit 848b0aea. ± Comparison against base commit 733373a0.

This pull request removes 9 and adds 2 tests. Note that renamed tests count towards both. ``` performance.coremark.test_coremark ‑ test_coremark performance.fibonacci.test_fibonacci ‑ test_fibonacci performance.psramspeed.test_psramspeed ‑ test_psramspeed performance.ramspeed.test_ramspeed ‑ test_ramspeed performance.superpi.test_superpi ‑ test_superpi validation.periman.test_periman ‑ test_periman validation.timer.test_timer ‑ test_timer validation.uart.test_uart ‑ test_uart validation.unity.test_unity ‑ test_unity ``` ``` validation.democfg.test_democfg ‑ test_cfg validation.wifi.test_wifi ‑ test_wifi ```

:recycle: This comment has been updated with latest results.

HamzaHajeir commented 1 month ago

Well spotted, missed to consider that path. Now should I add another commit or amend the commit?

On Fri, Oct 4, 2024, 11:48 TD-er @.***> wrote:

@.**** commented on this pull request.

In libraries/AsyncUDP/src/AsyncUDP.cpp https://github.com/espressif/arduino-esp32/pull/10415#discussion_r1787382785 :

@@ -473,12 +485,13 @@ bool AsyncUDP::_init() { if (_pcb) { return true; }

  • UDP_MUTEX_LOCK(); _pcb = udp_new(); if (!_pcb) { return false;

Shouldn't there be an unlock before the return?

β€” Reply to this email directly, view it on GitHub https://github.com/espressif/arduino-esp32/pull/10415#pullrequestreview-2347491418, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH3O7J2WS6AW55EUKMG375LZZZI63AVCNFSM6AAAAABPJS4YGOVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDGNBXGQ4TCNBRHA . You are receiving this because you authored the thread.Message ID: @.***>

me-no-dev commented 1 month ago

Now should I add another commit or amend the commit?

just add another commit to this PR

HamzaHajeir commented 1 month ago

just add another commit to this PR

Done

VojtechBartoska commented 1 month ago

Hello @HamzaHajeir, please sign CLA without that we are not able to merge this PR. Thanks!

HamzaHajeir commented 1 month ago

Hello @HamzaHajeir, please sign CLA without that we are not able to merge this PR. Thanks!

That bureaucratic :)

Couldn't find that document, can you guide how to?

TD-er commented 1 month ago

Hello @HamzaHajeir, please sign CLA without that we are not able to merge this PR. Thanks!

That bureaucratic :)

Couldn't find that document, can you guide how to?

See the 2nd post in this PR: https://github.com/espressif/arduino-esp32/pull/10415#issuecomment-2391191122

HamzaHajeir commented 1 month ago

Hello @HamzaHajeir, please sign CLA without that we are not able to merge this PR. Thanks!

That bureaucratic :) Couldn't find that document, can you guide how to?

See the 2nd post in this PR: #10415 (comment)

Thanks, done!