fix: too many software list requests

didier-wenzek commented 4 days ago

Proposed changes

One of the cause of too many software list requests, is that NanoMQ has not the same behavior as Mosquitto when a connection has overlapping subscriptions. See https://github.com/thin-edge/thin-edge.io/issues/2954#issuecomment-2188244048

[x] Improve how MQTT messages are logged
[x] Remove overlapping subscriptions on connect
[ ] Is there other differences in how NanoMQ behaves compared to Mosquitto?

Types of changes

[x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
[ ] Documentation Update (if none of the other choices apply)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

https://github.com/thin-edge/thin-edge.io/issues/2954

Checklist

[x] I have read the CONTRIBUTING doc
[x] I have signed the CLA (in all commits with git commit -s)
[x] I ran cargo fmt as mentioned in CODING_GUIDELINES
[x] I used cargo clippy as mentioned in CODING_GUIDELINES
[x] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)

Further comments

codecov[bot] commented 4 days ago

Codecov Report

Attention: Patch coverage is 68.29268% with 26 lines in your changes missing coverage. Please review.

Project coverage is 78.2%. Comparing base (5c3965e) to head (0b497e6). Report is 3 commits behind head on main.

Additional details and impacted files

| [Files](https://app.codecov.io/gh/thin-edge/thin-edge.io/pull/2964?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=thin-edge) | Coverage Δ | | |---|---|---| | [crates/common/mqtt\_channel/src/topics.rs](https://app.codecov.io/gh/thin-edge/thin-edge.io/pull/2964?src=pr&el=tree&filepath=crates%2Fcommon%2Fmqtt_channel%2Fsrc%2Ftopics.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=thin-edge#diff-Y3JhdGVzL2NvbW1vbi9tcXR0X2NoYW5uZWwvc3JjL3RvcGljcy5ycw==) | `92.1% <100.0%> (+5.9%)` | :arrow_up: | | [crates/extensions/tedge\_mqtt\_ext/src/lib.rs](https://app.codecov.io/gh/thin-edge/thin-edge.io/pull/2964?src=pr&el=tree&filepath=crates%2Fextensions%2Ftedge_mqtt_ext%2Fsrc%2Flib.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=thin-edge#diff-Y3JhdGVzL2V4dGVuc2lvbnMvdGVkZ2VfbXF0dF9leHQvc3JjL2xpYi5ycw==) | `71.1% <66.6%> (-2.1%)` | :arrow_down: | | [crates/core/tedge\_actors/src/message\_boxes.rs](https://app.codecov.io/gh/thin-edge/thin-edge.io/pull/2964?src=pr&el=tree&filepath=crates%2Fcore%2Ftedge_actors%2Fsrc%2Fmessage_boxes.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=thin-edge#diff-Y3JhdGVzL2NvcmUvdGVkZ2VfYWN0b3JzL3NyYy9tZXNzYWdlX2JveGVzLnJz) | `72.2% <0.0%> (-1.5%)` | :arrow_down: | | [crates/common/mqtt\_channel/src/messages.rs](https://app.codecov.io/gh/thin-edge/thin-edge.io/pull/2964?src=pr&el=tree&filepath=crates%2Fcommon%2Fmqtt_channel%2Fsrc%2Fmessages.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=thin-edge#diff-Y3JhdGVzL2NvbW1vbi9tcXR0X2NoYW5uZWwvc3JjL21lc3NhZ2VzLnJz) | `78.3% <0.0%> (-7.2%)` | :arrow_down: | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/thin-edge/thin-edge.io/pull/2964/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=thin-edge)

github-actions[bot] commented 4 days ago

Robot Results

:white_check_mark: Passed	:x: Failed	:next_track_button: Skipped	Total	Pass %	:stopwatch: Duration
454	0	3	454	100	57m34.186704s

reubenmiller commented 3 days ago

@didier-wenzek I just re-ran the original setup using the tedge build 1.1.2~144+g0b497e6, and there are still multiple software_list operation being created by the tedge-mapper-c8y (see logs below):

https://github.com/thin-edge/debug-tedge-readonly/tree/main/logs/run3-g0b497e6

And another run with the log levels set to DEBUG (so it should be much more helpful):

https://github.com/thin-edge/debug-tedge-readonly/tree/main/logs/run4-g0b497e6

thin-edge / thin-edge.io