mar-file-system / GUFI

Grand Unified File-Index
Other
46 stars 23 forks source link

Verbose stats and verbose error message #142

Closed johnbent closed 1 year ago

johnbent commented 1 year ago

Two small changes here:

  1. I always get confused when I don't have a config in /etc/GUFI and I'm not running my modified version which allows configurable values of gufi config path. This change prints an explanatory error message so it will be clear when other people (including myself) run without a gufi config in /etc/GUFI
  2. Added verbose mode to gufi_stats to print the actual gufi_query. Useful for debugging and for people learning how it works.
codecov-commenter commented 1 year ago

Codecov Report

All modified lines are covered by tests :white_check_mark:

Comparison is base (8972468) 91.33% compared to head (68978e8) 91.37%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #142 +/- ## ========================================== + Coverage 91.33% 91.37% +0.04% ========================================== Files 58 58 Lines 6542 6564 +22 Branches 745 744 -1 ========================================== + Hits 5975 5998 +23 + Misses 368 367 -1 Partials 199 199 ``` | [Files](https://app.codecov.io/gh/mar-file-system/GUFI/pull/142?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Ξ” | | |---|---|---| | [scripts/gufi\_common.py](https://app.codecov.io/gh/mar-file-system/GUFI/pull/142?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c2NyaXB0cy9ndWZpX2NvbW1vbi5weQ==) | `95.29% <100.00%> (+0.62%)` | :arrow_up: | | [scripts/gufi\_config.py](https://app.codecov.io/gh/mar-file-system/GUFI/pull/142?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c2NyaXB0cy9ndWZpX2NvbmZpZy5weQ==) | `96.59% <100.00%> (ΓΈ)` | | | [scripts/gufi\_find](https://app.codecov.io/gh/mar-file-system/GUFI/pull/142?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c2NyaXB0cy9ndWZpX2ZpbmQ=) | `97.76% <100.00%> (+0.01%)` | :arrow_up: | | [scripts/gufi\_getfattr](https://app.codecov.io/gh/mar-file-system/GUFI/pull/142?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c2NyaXB0cy9ndWZpX2dldGZhdHRy) | `97.29% <100.00%> (+0.07%)` | :arrow_up: | | [scripts/gufi\_ls](https://app.codecov.io/gh/mar-file-system/GUFI/pull/142?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c2NyaXB0cy9ndWZpX2xz) | `97.69% <100.00%> (+0.03%)` | :arrow_up: | | [scripts/gufi\_stat](https://app.codecov.io/gh/mar-file-system/GUFI/pull/142?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c2NyaXB0cy9ndWZpX3N0YXQ=) | `93.93% <100.00%> (+0.83%)` | :arrow_up: | | [scripts/gufi\_stats](https://app.codecov.io/gh/mar-file-system/GUFI/pull/142?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-c2NyaXB0cy9ndWZpX3N0YXRz) | `98.03% <100.00%> (+0.01%)` | :arrow_up: | ... and [2 files with indirect coverage changes](https://app.codecov.io/gh/mar-file-system/GUFI/pull/142/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

johnbent commented 1 year ago

OK. I believe I correctly everything requested. Please LMK. Thanks for great feedback!

calccrypto commented 1 year ago

Regression tests also need updating

johnbent commented 1 year ago

Cool. All tests passed! I might actually be getting better at this whole git/github thing!

garygrider commented 1 year ago

Is it a thead thing?

From: John Bent @.> Reply-To: mar-file-system/GUFI @.> Date: Thursday, October 5, 2023 at 8:47 PM To: mar-file-system/GUFI @.> Cc: Subscribed @.> Subject: [EXTERNAL] Re: [mar-file-system/GUFI] Verbose stats and verbose error message (PR #142)

@johnbent commented on this pull request.


In scripts/gufi_stathttps://urldefense.com/v3/__https:/github.com/mar-file-system/GUFI/pull/142*discussion_r1348189042__;Iw!!Bt8fGhp8LhKGRg!AWSbL2p7cEdx_LMkbcZeA1z1RvXCvYUogioJFZ4kfibr8QbGRG-uDLmFcq55h1uq-B73VSpdQEnSK29G-_zKjFG2$:

@@ -133,6 +139,9 @@ def run(argv, config_path):

 paths = [os.path.normpath(os.path.sep.join([config.indexroot(), path]))

          for path in args.paths]

@calccryptohttps://urldefense.com/v3/__https:/github.com/calccrypto__;!!Bt8fGhp8LhKGRg!AWSbL2p7cEdx_LMkbcZeA1z1RvXCvYUogioJFZ4kfibr8QbGRG-uDLmFcq55h1uq-B73VSpdQEnSK29G-yMnAyeM$ , here is where the verbose print is happening in gufi_stat. It's before the call to subprocess so I'm confused as to why the verbose is printed after the output. When I run ./test/regression/gufi_stat.shhttps://urldefense.com/v3/__http:/gufi_stat.sh__;!!Bt8fGhp8LhKGRg!AWSbL2p7cEdx_LMkbcZeA1z1RvXCvYUogioJFZ4kfibr8QbGRG-uDLmFcq55h1uq-B73VSpdQEnSK29G-1UWtDhZ$, it does show it backwards with the output first and then the verbose printout. But when I run the command at the command line, it shows the verbose printout before the output as expected. So, I'll keep looking but this is confusing.

β€” Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/mar-file-system/GUFI/pull/142*pullrequestreview-1661037349__;Iw!!Bt8fGhp8LhKGRg!AWSbL2p7cEdx_LMkbcZeA1z1RvXCvYUogioJFZ4kfibr8QbGRG-uDLmFcq55h1uq-B73VSpdQEnSK29G-9VqzCNp$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ACZXDGOFORVQAS3ELHG6CUDX55WKNAVCNFSM6AAAAAA5SKGROGVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTMNRRGAZTOMZUHE__;!!Bt8fGhp8LhKGRg!AWSbL2p7cEdx_LMkbcZeA1z1RvXCvYUogioJFZ4kfibr8QbGRG-uDLmFcq55h1uq-B73VSpdQEnSK29G-9N8B2Lz$. You are receiving this because you are subscribed to this thread.Message ID: @.***>

johnbent commented 1 year ago

Yeah @garygrider , it was maybe sort of a thread thing. The parent was calling print which is buffered and then the parent was calling subprocess which makes a child and that child did some output which was showing up before the parent's print. But not when the parent was called directly from the command line but only when the script was called by another script. A bit weird but apparently, "This buffering is especially apparent when Python's output is redirected, which may happen when you run the code from another script."

garygrider commented 1 year ago

My guess is its python 😊

From: John Bent @.> Reply-To: mar-file-system/GUFI @.> Date: Thursday, October 5, 2023 at 9:19 PM To: mar-file-system/GUFI @.> Cc: Gary Grider @.>, Mention @.***> Subject: [EXTERNAL] Re: [mar-file-system/GUFI] Verbose stats and verbose error message (PR #142)

@johnbent commented on this pull request.


In test/regression/gufi_stat.expectedhttps://urldefense.com/v3/__https:/github.com/mar-file-system/GUFI/pull/142*discussion_r1348201809__;Iw!!Bt8fGhp8LhKGRg!DP4FfUnzKchVW24a_-6V4FilsSedR3TSqtjXE6a-DHsnKaLujIZfpRoJES_uUqWIMLSiWJL-KIIymAbHZje_y89i$:

+'prefix/.hidden' prefix/.hidden 664 -rw-rw-r-- ? ? 81b4 regular file 9 - 0 1970-01-01 00:00:09 +0000 9 1970-01-01 00:00:09 +0000 9

+GUFI query is

So weird. I just added sys.stdout.flush() to gufi_common.pyhttps://urldefense.com/v3/__http:/gufi_common.py__;!!Bt8fGhp8LhKGRg!DP4FfUnzKchVW24a_-6V4FilsSedR3TSqtjXE6a-DHsnKaLujIZfpRoJES_uUqWIMLSiWJL-KIIymAbHZur5c5gw$ and now suddenly a bunch of tests are failing. I'm looking into it but I'm not sure what's causing these errors.

β€” Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/mar-file-system/GUFI/pull/142*discussion_r1348201809__;Iw!!Bt8fGhp8LhKGRg!DP4FfUnzKchVW24a_-6V4FilsSedR3TSqtjXE6a-DHsnKaLujIZfpRoJES_uUqWIMLSiWJL-KIIymAbHZje_y89i$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ACZXDGP5KYB4ANMKA25NUJDX552DLAVCNFSM6AAAAAA5SKGROGVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTMNRRGA2TINBXGA__;!!Bt8fGhp8LhKGRg!DP4FfUnzKchVW24a_-6V4FilsSedR3TSqtjXE6a-DHsnKaLujIZfpRoJES_uUqWIMLSiWJL-KIIymAbHZsIKAkcD$. You are receiving this because you were mentioned.Message ID: @.***>