abrt / retrace-server

Application for remote coredump analysis
GNU General Public License v2.0
40 stars 30 forks source link

retrace: Re-use ProcessCommunicateTimeout with makedumpfile #456

Closed DaveWysochanskiRH closed 2 years ago

DaveWysochanskiRH commented 2 years ago

As https://bugzilla.redhat.com/show_bug.cgi?id=2049284 explains, makedumpfile can infinite loop similar to what has previously happened with crash. With makedumpfile, such a loop can have impacts beyond just CPU usage, and as was seen in the above bug, all free disk space may fill up and thus have larger impacts.

To resolve this issue, first, re-use the same config setting, CONFIG["ProcessCommunicateTimeout"], for makedumpfile as we did for crash in commit becb40663a003. Second, increase the default value of CONFIG["ProcessCommunicateTimeout"] to reflect a more realistic "worst case" valid runtime. In production we have seen makedumpfile may run up to 6 hours on some larger vmcores. Given various filesystems performance characteristics and to add some margin, set the default of CONFIG["ProcessCommunicateTimeout"] to 43200 seconds (12 hours). Finally, improve logging to reflect both when makedumpfile starts running as well as if a timeout occurs.

Resolves: https://github.com/abrt/retrace-server/issues/455 Signed-off-by: Dave Wysochanski dwysocha@redhat.com

codecov-commenter commented 2 years ago

Codecov Report

Merging #456 (d1f4507) into master (3f52f4d) will decrease coverage by 0.11%. The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #456      +/-   ##
==========================================
- Coverage   20.81%   20.70%   -0.12%     
==========================================
  Files          14       14              
  Lines        2796     2811      +15     
==========================================
  Hits          582      582              
- Misses       2214     2229      +15     
Flag Coverage Δ
unittests 20.70% <0.00%> (-0.12%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/retrace/hooks/hooks.py 27.63% <ø> (ø)
src/retrace/retrace.py 16.51% <0.00%> (-0.17%) :arrow_down:
src/retrace/retrace_worker.py 6.24% <0.00%> (ø)
src/retrace/util.py 33.84% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 3f52f4d...d1f4507. Read the comment docs.