search

datalad / datalad-fuse

DataLad extension to provide FUSE file system access
Other
1 stars 4 forks source link

Get size of annexed files from keys where possible #86

Closed jwodder closed 1 year ago

jwodder commented 1 year ago

Closes #84.

codecov[bot] commented 1 year ago

Codecov Report

Base: 71.06% // Head: 71.51% // Increases project coverage by +0.44% :tada:

Coverage data is based on head (8412c61) compared to base (5421d3a). Patch coverage: 70.32% of modified lines in pull request are covered.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #86 +/- ## ========================================== + Coverage 71.06% 71.51% +0.44% ========================================== Files 11 11 Lines 788 839 +51 ========================================== + Hits 560 600 +40 - Misses 228 239 +11 ``` | [Impacted Files](https://codecov.io/gh/datalad/datalad-fuse/pull/86?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad) | Coverage Δ | | |---|---|---| | [datalad\_fuse/fuse\_.py](https://codecov.io/gh/datalad/datalad-fuse/pull/86/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9mdXNlL2Z1c2VfLnB5) | `25.98% <8.69%> (-0.34%)` | :arrow_down: | | [datalad\_fuse/fsspec.py](https://codecov.io/gh/datalad/datalad-fuse/pull/86/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9mdXNlL2Zzc3BlYy5weQ==) | `83.12% <70.00%> (-1.35%)` | :arrow_down: | | [datalad\_fuse/utils.py](https://codecov.io/gh/datalad/datalad-fuse/pull/86/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9mdXNlL3V0aWxzLnB5) | `95.83% <94.33%> (-4.17%)` | :arrow_down: | | [datalad\_fuse/tests/test\_fuse.py](https://codecov.io/gh/datalad/datalad-fuse/pull/86/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9mdXNlL3Rlc3RzL3Rlc3RfZnVzZS5weQ==) | `100.00% <100.00%> (ø)` | | | [datalad\_fuse/tests/test\_util.py](https://codecov.io/gh/datalad/datalad-fuse/pull/86/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad#diff-ZGF0YWxhZF9mdXNlL3Rlc3RzL3Rlc3RfdXRpbC5weQ==) | `100.00% <100.00%> (ø)` | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

yarikoptic commented 1 year ago

I am curious -- have you tried this branch on 000026 in https://github.com/dandi/dandisets-healthstatus -- did it provide remedy for the slow "traversal"?

jwodder commented 1 year ago

@yarikoptic I'd rather not try the healthcheck on this unless #83 was merged in so I could rebase on top of it.

yarikoptic commented 1 year ago

@yarikoptic I'd rather not try the healthcheck on this unless #83 was merged in so I could rebase on top of it.

take #83 out of draft? ;-)

jwodder commented 1 year ago

@yarikoptic Traversing 000026 using this branch now takes about 2 or 3 minutes (I don't have an exact time).

yarikoptic commented 1 year ago

@yarikoptic Traversing 000026 using this branch now takes about 2 or 3 minutes (I don't have an exact time).

ok, not super fast but much better than before and given number of files -- not too bad really. Would be worth py-spy top'ing it to see where time is spent. Let's proceed with this as already significant improvement.