irmin-pack: add GC design docs

metanivek commented 1 year ago

This PR adds design docs for different phases of GC work.

adatario commented 1 year ago

Nit: Can you add the figures to the repo? It would be nice if the documentation is self-contained and does not rely on external resources from imgur.

samoht commented 1 year ago

Another nit: probably best not to add a pdf file to the repo.

metanivek commented 1 year ago

@adatario yeah, I agree. I was mostly just being lazy but will take care of it. :sweat_smile: (/me grumbles about hackmd not having a good export...)

@samoht this pdf was copied from https://github.com/tomjridge/public_notes/blob/master/2022-02-03_layered_store_design.pdf. I like not relying on another source, but if we don't want it in the repo I could just link to it (unless somebody feels excited to re-do the doc in a text-based format!).

samoht commented 1 year ago

I've pushed a markdownified version of the layered store design.

codecov-commenter commented 1 year ago

Codecov Report

Merging #2228 (ccab23b) into main (8de16b8) will decrease coverage by 0.12%. The diff coverage is 33.33%.

:exclamation: Current head ccab23b differs from pull request most recent head 33a4666. Consider uploading reports for the commit 33a4666 to get more accurate results

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             main    #2228      +/-   ##
==========================================
- Coverage   68.09%   67.98%   -0.12%     
==========================================
  Files         136      136              
  Lines       16506    16560      +54     
==========================================
+ Hits        11240    11258      +18     
- Misses       5266     5302      +36

Impacted Files	Coverage Δ
src/irmin-pack/unix/errors.ml	`28.88% <ø> (ø)`
src/irmin-pack/unix/file_manager.ml	`84.69% <ø> (ø)`
src/irmin-pack/unix/gc_stats.ml	`31.14% <0.00%> (+0.50%)`	:arrow_up:
src/irmin-pack/unix/gc_worker.ml	`3.59% <0.00%> (-0.67%)`	:arrow_down:
src/irmin-pack/unix/io_errors.ml	`58.33% <ø> (ø)`
src/irmin-pack/unix/lower.ml	`60.84% <0.00%> (-2.68%)`	:arrow_down:
src/irmin-pack/unix/lower_intf.ml	`0.00% <ø> (ø)`
src/irmin-pack/unix/checks.ml	`23.44% <38.46%> (+0.15%)`	:arrow_up:
src/irmin/store.ml	`66.23% <65.71%> (+<0.01%)`	:arrow_up:
src/irmin/conf.ml	`75.36% <75.00%> (-0.05%)`	:arrow_down:
... and 3 more

... and 2 files with indirect coverage changes

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

metanivek commented 1 year ago

Thanks @samoht! I'll update my commit next week to remove the pdf and add the images.

metanivek commented 1 year ago

All updated with in-repo images now @adatario @samoht!

For reference, I used this little script to automate the extraction/replacement of the images. It could be useful for future additions/updates if they originate in hackmd. I could add it to the repo, add it to the readme, or just leave it as a comment here. Any opinions?

#!/usr/bin/env sh
set -e

FILE=$(basename "$1")
LOC=$(basename "$FILE" .md)
DIR="imgs/$LOC"

echo "Make dir $DIR"
mkdir -p "$DIR"

echo "Download images from $FILE"
grep -oP "https://.*imgur.*png" "$FILE" | xargs wget -P "$DIR"

echo "Replace links in $FILE"
sed -i "s,https://i\.imgur\.com,./$DIR,g" "$FILE"

echo "Done"

mirage / irmin

irmin-pack: add GC design docs #2228

Codecov Report