BallAerospace / COSMOS

Ball Aerospace COSMOS
https://ballaerospace.github.io/cosmos-website/
Other
360 stars 129 forks source link

Logging Optimizations #1714

Closed ryanmelt closed 2 years ago

ryanmelt commented 2 years ago

Make all packet log writers share cycle thread. With large numbers of packets, this makes a huge difference in the number of threads needed to run COSMOS.

Default packet buffer to nil. This improves everywhere where a System is generated. Packets will not allocate space for their buffers until needed. Will greatly reduce RAM allocation in situations where there are large numbers of defined but unused packets.

Fix UDP multicast test on Mac. Macs do not like Multicast specifying port 0.

codecov[bot] commented 2 years ago

Codecov Report

Merging #1714 (09f3962) into master (69ef6a8) will decrease coverage by 0.05%. The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1714      +/-   ##
==========================================
- Coverage   78.22%   78.17%   -0.06%     
==========================================
  Files         235      236       +1     
  Lines       18188    18251      +63     
==========================================
+ Hits        14228    14267      +39     
- Misses       3960     3984      +24     
Impacted Files Coverage Δ
...b/cosmos/tools/table_manager/table_manager_core.rb 88.46% <0.00%> (-10.00%) :arrow_down:
lib/cosmos/packets/structure.rb 95.79% <0.00%> (-2.32%) :arrow_down:
config/routes.rb 100.00% <0.00%> (ø)
app/controllers/completed_script_controller.rb 100.00% <0.00%> (ø)
app/models/table.rb 90.98% <0.00%> (ø)
lib/cosmos/logs/log_writer.rb 98.47% <0.00%> (+0.25%) :arrow_up:
lib/cosmos/tools/table_manager/table_config.rb 93.80% <0.00%> (+0.60%) :arrow_up:
app/controllers/application_controller.rb 100.00% <0.00%> (+11.11%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 69ef6a8...09f3962. Read the comment docs.

ryanmelt commented 2 years ago

The difference this makes is huge. My CPUs no longer have any stress.

ryanmelt commented 2 years ago

For the specific use case we've been looking at, it reduced the number of threads from more than the system could handle (100,000+) down to 110. CPU utilization seems to be about 75% less as well, but that is hard to really quantify as it couldn't handle it before.