abrt / retrace-server

Application for remote coredump analysis
GNU General Public License v2.0
40 stars 30 forks source link

Add logic to determine if we save a stripped vmcore #452

Closed audrabaker closed 2 years ago

audrabaker commented 2 years ago

Currently we do not check to see if running makedumpfile and creating a stripped vmcore is actually worthwhile. In some situations we may see little to no benefit to stripping down a vmcore and replacing the original with the newly stripped vmcore. For those situation, this patch adds the config variable SpaceSavePercent which is meant to indicate the percentage of space a stripped vmcore will need to save for us to choose to save it in place of the original. The proposed default is 10%. This patch also adjusts the logic so that the relevant message about makedumpfile will be printed (i.e. if makedumpfile is not run or we don't use the stripped vmcore from makedumpfile we do not print how much space we saved).

Signed-off-by: Audra Mitchell aubaker@redhat.com #445

DaveWysochanskiRH commented 2 years ago

@audrabaker some feedback.

  1. commit message needs limited to some reasonable # of columns (70?). Do a 'git show' on other commits and you'll get an idea. It shows as one very long line.

    $ git show audrabaker/master
    commit f1e647b28389b793ce54425d306e0dda150786af
    Author: Audra Mitchell <aubaker@redhat.com>
    Date:   Tue Jan 18 15:43:26 2022 -0500
    
    Add logic to determine if we save a stripped vmcore
    
    Currently we do not check to see if running makedumpfile and creating a stripped vmcore is actually worthwhile. In some situations we may see little to no benefit to stripping down a vmcore and 
    
    Signed-off-by: Audra Mitchell <aubaker@redhat.com>
  2. I think the name of SpaceSavePercent is ok but probably needs improved. We probably need vmcore in there because this only applies to vmcores. Also, it's makedumpfile related so I wonder if we can clue in there too. The only other parameter though is "VmcoreDumpLevel" so maybe "VmcoreDumpSavePercent" or is a little long but at least it's grouped somewhat with the other name.

  3. Suggest moving the following code inside vmcore.strip_extra_pages since inside that function you're logging other things and you've got the sizes there.

    +            if vmcore.strip_extra_pages():
    +                dur = int(time.time() - start)
    +                newsize = vmcore_path.stat().st_size
    +                log_info("Stripped size: %s" % human_readable_size(newsize))
    +                log_info("Makedumpfile took %d seconds and saved %s"
                      % (dur, human_readable_size(oldsize - newsize)))
  4. Another thought I had was is there any point in running makedumpfile if the vmcore is smaller than a certain size, but I'm not suggesting you do it. I think not but I don't have a good solution for that and don't really think yet another config variable is a good idea. Maybe a hardcoded minimum but maybe that's not worthwhile either. Alternatively, it might be worthwhile to have a button to avoid makedumpfile on a vmcore but that's a separate discussion / patch.

Let me do some testing and see if there's anything else I see and let you know.

DaveWysochanskiRH commented 2 years ago

If I'm understanding "SpaceSavePercent" correctly, it means "if we save at least this much space, we want to keep the stripped vmcore, otherwise we don't". If so, then the logic is reversed, and you want < here:

         elif ((oldsize - newvmcore.stat().st_size) / int(oldsize) * 100 > CONFIG["SpaceSavePercent"]):
DaveWysochanskiRH commented 2 years ago

You might want to improve this message to print the value of SpaceSavePercent (or whatever the final name is)

[2022-01-19 06:59:38] [W] makedumpfile did not strip enough pages. Not saving new vmcore.

Maybe something like

[2022-01-19 06:59:38] [W] makedumpfile did not strip enough pages (needed 10%). Not saving new vmcore.
DaveWysochanskiRH commented 2 years ago

@audrabaker you can also add this to the bottom near the signature: Resolves: https://github.com/abrt/retrace-server/issues/445

DaveWysochanskiRH commented 2 years ago

You'll need to rebase this too and re-push due to changes to resolve https://github.com/abrt/retrace-server/issues/455

lgtm-com[bot] commented 2 years ago

This pull request introduces 2 alerts when merging f87f673cbc4aaf04b4c9683675b19bdc4c827a8e into b564b552e82182caae3a662e8ca81c0063725dea - view on LGTM.com

new alerts:

lgtm-com[bot] commented 2 years ago

This pull request introduces 2 alerts when merging ed0289f90ecd7254e5e4c42a1182531811635832 into b564b552e82182caae3a662e8ca81c0063725dea - view on LGTM.com

new alerts:

lgtm-com[bot] commented 2 years ago

This pull request introduces 2 alerts when merging c202a66cb05d3153a88d3c6af2aca19afafa7427 into aa66f3a1a9dffd09671c7652513bc1108d5f23b5 - view on LGTM.com

new alerts:

DaveWysochanskiRH commented 2 years ago

@mgrabovsky what do you think?

lgtm-com[bot] commented 2 years ago

This pull request introduces 2 alerts when merging ed0289f90ecd7254e5e4c42a1182531811635832 into aa66f3a1a9dffd09671c7652513bc1108d5f23b5 - view on LGTM.com

new alerts:

codecov-commenter commented 2 years ago

Codecov Report

Merging #452 (f4111cf) into master (b564b55) will increase coverage by 1.27%. The diff coverage is 9.09%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #452      +/-   ##
==========================================
+ Coverage   20.70%   21.98%   +1.27%     
==========================================
  Files          14       18       +4     
  Lines        2811     2848      +37     
==========================================
+ Hits          582      626      +44     
+ Misses       2229     2222       -7     
Flag Coverage Δ
unittests 21.98% <9.09%> (+1.27%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/retrace/retrace_worker.py 6.56% <ø> (+0.32%) :arrow_up:
src/retrace/retrace.py 17.09% <9.09%> (+0.58%) :arrow_up:
test/test_util.py 100.00% <0.00%> (ø)
src/retrace/stats.py 14.45% <0.00%> (ø)
test/test_backends.py 100.00% <0.00%> (ø)
src/retrace/__init__.py 100.00% <0.00%> (ø)
src/retrace/argparser.py 36.84% <0.00%> (ø)
test/test_architecture.py 100.00% <0.00%> (ø)
src/retrace/archive.py 16.76% <0.00%> (ø)
src/retrace/logging.py 75.00% <0.00%> (ø)
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update b564b55...f4111cf. Read the comment docs.

lgtm-com[bot] commented 2 years ago

This pull request introduces 1 alert when merging f4111cf09c5cc59fdc79c1a93e1a688506ce6df6 into 2c0305a3176f6ec53e8bd784a5bb35a0d9b56bf6 - view on LGTM.com

new alerts:

DaveWysochanskiRH commented 2 years ago

@audrabaker looks like you need to squash the two commits into one, and re-push a single commit:

$ git log --oneline audrabaker/master | head
f4111cf09c5c Add logic to determine if we save a stripped vmcore
ed0289f90ecd Add logic to determine if we save a stripped vmcore
b564b552e821 github: CD to build directory for checks
mgrabovsky commented 2 years ago

There's an unused variable oldsize left on line 793 of src/retrace/retrace_worker.py after the refactoring. Once that's removed and squashed, I think we're good to merge.

audrabaker commented 2 years ago

Happy to help!