Closed jaber-the-great closed 1 year ago
That's a terrifyingly big list of changed files? Is there a merge in there, or does it need to be retargeted to develop, perhaps?
On Thu, Aug 3, 2023, 6:05 PM jaber daneshamooz @.***> wrote:
Problem: The issue with number of available queues in LibreQoS.py. In line 703 and 724, the for loops iterate through the number of available queues and create MQ qdisc for each CPU core / rx-tx queue. The issue is that the function findQueuesAvailable() only counts the number of tx queues for interfaceA (Interface connected to core router) and do not separately count the number of tx queues for interfaceB(Interface connected to edge router).
The code would work fine if: number of tx queues for interfaceB <= number of tx queues for interfaceA or number of tx queues for interfaceB <= cpuCount
This scenario may not always be true and may raise error in the for loop of line 724.
Fixes:
- Changing findQueuesAvailable() : The function now gets the interface name as input and finds the number of tx queues based on that. Previous version works on InterfaceA by defualt. Also, changing the error and logging messages to print the interface name.
- Handling the case when the available queues for interface A and B are different: Choosing the minimum one (in order to not mess up with assigning parent nodes to different classes in major:minor format)
You can view, comment on, or merge this pull request online at:
https://github.com/LibreQoE/LibreQoS/pull/376 Commit Summary
- 72e407e https://github.com/LibreQoE/LibreQoS/pull/376/commits/72e407e1dae083247db65ee1aa99661fadb5b773 Add infrastructure to the generated topology. Commit for testing.
- da16691 https://github.com/LibreQoE/LibreQoS/pull/376/commits/da16691306a8a3baf7a011ae403255ffe10784c8 Merge branch 'long_term_stats' of https://github.com/LibreQoE/LibreQoS into long_term_stats
- acabdc7 https://github.com/LibreQoE/LibreQoS/pull/376/commits/acabdc7ff25f5be58cb711e5f4494ced2a93bd28 Re-add checked arithmetic to the packet counter. It may not solve the spike problem, but it should be there anyway.
- 4a1e7e7 https://github.com/LibreQoE/LibreQoS/pull/376/commits/4a1e7e7eb75bf1f880c2fdfeb81fd6aefd080491 Another consistency update - the iterator for adding data was missing a filter we use elsewhere.
- eb0585d https://github.com/LibreQoE/LibreQoS/pull/376/commits/eb0585d275457c57a1c215e18fd8b4542548b73a Unknown devices download in node manager includes bandwidth used and a header line.
- 8636ded https://github.com/LibreQoE/LibreQoS/pull/376/commits/8636ded82bcda32721f430d51f012d3ccf78ab72 Some more attempts at handling total graph spikes. Much stricter handling of new flows.
- de0fec0 https://github.com/LibreQoE/LibreQoS/pull/376/commits/de0fec023251557210b2c2944cd3d22fe3e49ceb Add support for "appendToShapedDevices.csv".
- 9772c08 https://github.com/LibreQoE/LibreQoS/pull/376/commits/9772c08ac8e62d6b63a4f7e5bd9599aff731df6c Add error guards to tell me if the collection cycle took too long, reduce tokio blocking load by moving the collection calculations into their own blocking task.
- bf95176 https://github.com/LibreQoE/LibreQoS/pull/376/commits/bf951762466f53987bdf109343dde3424652dbae Move stat submission to a separate tokio task, fixing the time overflow issue.
- 343ebca https://github.com/LibreQoE/LibreQoS/pull/376/commits/343ebca557bf52669e81a6a2c3e7c7504aa02bfd Undo the 'all devices, everywhere' - it was causing problems
- 5afbd1b https://github.com/LibreQoE/LibreQoS/pull/376/commits/5afbd1bff51dfdf2df5e86cc40f8fff4a79518f1 Separate the 'are we connected' timer from the regular tick
- 9d5f9a3 https://github.com/LibreQoE/LibreQoS/pull/376/commits/9d5f9a3e73fd845063089a531364bbfefe02545b Implement an actually sane queue system for WSS messaging, with re-connect baked in.
- 1897872 https://github.com/LibreQoE/LibreQoS/pull/376/commits/1897872852b4d5316aec56dc6d0a2b9a7ad82a6d A couple of Rust style improvements.
- 484da39 https://github.com/LibreQoE/LibreQoS/pull/376/commits/484da39ddf54adebe7dce065acd23a6bf8e62248 Merge pull request #371 from LibreQoE/long_term_stats
- a8302a1 https://github.com/LibreQoE/LibreQoS/pull/376/commits/a8302a168ca188539d52b18fbc23c26b09d4793d Update README.md
- 5000eab https://github.com/LibreQoE/LibreQoS/pull/376/commits/5000eab638f2a8d84fe9de605e291425c7dd245d Update README.md
- ea883ae https://github.com/LibreQoE/LibreQoS/pull/376/commits/ea883aebb8592d2136383005ad81ffcb2ff1a3db Catch None type in integrationUISP
- fc7dd8a https://github.com/LibreQoE/LibreQoS/pull/376/commits/fc7dd8aeb0a4cd384198ce3f3906618fac25a51b Solve PtMP issue bug
- 21e6297 https://github.com/LibreQoE/LibreQoS/pull/376/commits/21e6297df03b4ea28b92495a0ee4817e5fb63227 Update troubleshooting.md
- 1820767 https://github.com/LibreQoE/LibreQoS/pull/376/commits/1820767d3105d5bf8e64a16b2ad87166b2ddf130 Handling the case when the number of TX queues per interface are different
File Changes
(21 files https://github.com/LibreQoE/LibreQoS/pull/376/files)
- M README.md https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5 (9)
- M docs/TechnicalDocs/troubleshooting.md https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-f08aa3c350368969250b875172329c38c180b55d40bddf3e43dd6b88bc663391 (5)
- M src/LibreQoS.py https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-dbb21a7be660db7d6e78f8bd86f3c885e6df990f313d99f48cfa1d827d31d8a2 (19)
- M src/integrationCommon.py https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-5bc3e8b2513f55d46de387d5bfd3c93538fafa08103f5d2421e6fe9feee55db9 (16)
- M src/integrationUISP.py https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-7a07ecf02cf5bb38250dd70748b865e97cdc145230df961c3b880699de1c24e0 (12)
- M src/rust/long_term_stats/lts_node/run.sh https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-99414f052d89e3c4f08af2390b7b0d03e85daf41004d476ee8743ed52d3b68f5 (2)
- M src/rust/long_term_stats/lts_node/src/web/wss/queries/rtt/mod.rs https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-e9802febe961e7212f981b7bd6484a725ee5b2b8e68d4cfc85dbbe2224f9cf1b (4)
- M src/rust/long_term_stats/lts_node/web/app.js https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-61e807ca687b6326de50824292db1e80a677cdb35e10a964133b4ca0fa819a18 (258)
- M src/rust/long_term_stats/lts_node/web/app.js.map https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-3d1540dd4aced76f2793929a25468287db9a4f7961446416244e0837e0993a2f (6)
- M src/rust/long_term_stats/lts_node/web/wasm_pipe_bg.wasm https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-3a33f9db633c0a5991b51174dcffe55de272555023ad88b337c71600428bd5b0 (0)
- M src/rust/long_term_stats/site_build/src/app.ts https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-e79d7b8dd2948dbb80015bea7ed6d950d5f65599eb41c901c72bfe5aeb349f79 (17)
- M src/rust/long_term_stats/site_build/src/bus.ts https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-8244c355bbf189d9a7eac0f2a1fe1cc11920adfd246176e7776b9b0c28b6c834 (8)
- M src/rust/long_term_stats/site_build/wasm/wasm_pipe.d.ts https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-2bc92d54b00d6f64cd5ec24add4743ad6a401405fe176a7fde76c1246029d264 (7)
- M src/rust/long_term_stats/site_build/wasm/wasm_pipe.js https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-a119b97e270c9d91075b2ea3ac31bb53f5931ba6c0c5f9cef659f7f93b81cffe (244)
- M src/rust/long_term_stats/site_build/wasm/wasm_pipe_bg.wasm https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-35093452100fccd89d28fced553d22efef3ba6a28ea100af38166c8ca4e8dd5d (0)
- M src/rust/long_term_stats/site_build/wasm/wasm_pipe_bg.wasm.d.ts https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-c27afd93b380a0b9e7ac171618257c86cf9bc6a0b54aca48ca6827c94f97d430 (4)
- M src/rust/long_term_stats/wasm_pipe/build.sh https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-60f52c8cf1d17225a757a195ded4bb9609463ba30a4603b400b1a03cde867225 (4)
- M src/rust/long_term_stats/wasm_pipe/src/lib.rs https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-ea4a091146329a8ac4dd7b1eaace9ee2f7c91cd2eed74315842c4dfda73f80a8 (35)
- M src/rust/lqos_node_manager/src/unknown_devices.rs https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-b1bec86ea880d9fa8ad43e7790455e0287b71bce4a1dc9807ec1b87e06aa5fa4 (4)
- M src/rust/lqosd/src/throughput_tracker/mod.rs https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-d0f91ec64e4716341322403f9cf21968fa288041d3210fea9abcf18c8a16aae2 (35)
- M src/rust/lqosd/src/throughput_tracker/tracking_data.rs https://github.com/LibreQoE/LibreQoS/pull/376/files#diff-dec08cd160f7ab8b02f182350a854533893ecd8675824c7c5f2a8442297f7185 (21)
Patch Links:
- https://github.com/LibreQoE/LibreQoS/pull/376.patch
- https://github.com/LibreQoE/LibreQoS/pull/376.diff
— Reply to this email directly, view it on GitHub https://github.com/LibreQoE/LibreQoS/pull/376, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADRU433V66VF53KYD4ABGWLXTQOBPANCNFSM6AAAAAA3DMETQY . You are receiving this because you are subscribed to this thread.Message ID: @.***>
Oops. My bad, merge problem. Issued another pull request.
Problem: The issue with number of available queues in LibreQoS.py. In line 703 and 724, the for loops iterate through the number of available queues and create MQ qdisc for each CPU core / rx-tx queue. The issue is that the function findQueuesAvailable() only counts the number of tx queues for interfaceA (Interface connected to core router) and do not separately count the number of tx queues for interfaceB(Interface connected to edge router).
The code would work fine if: number of tx queues for interfaceB <= number of tx queues for interfaceA or number of tx queues for interfaceB <= cpuCount
This scenario may not always be true and may raise error in the for loop of line 724.
Fixes: