New zimgit ZIM significantly smaller

[Popolechien]

I just created https://download.kiwix.org/zim/other/zimgit-food-preparation_en_2023-06.zim and noticed that it is markedly smaller than https://download.kiwix.org/zim/other/zimgit-food-preparation_en_2022-03.zim (92 Mb vs 203 MB). Yet as far as I can tell both files have identical contents.

Same thing with other zimgit files in other that were generated the same day (but the difference is less dramatic). Any major update here that happened over the past couple of month?

[rgaudin]

❯ zimls zimgit-food-preparation_en_2023-06.zim |grep files
208.37 KB files/Food Preparation (1).pdf
2.21 MB files/Food Preparation (10).pdf
1.45 MB files/Food Preparation (11).pdf
3.17 MB files/Food Preparation (12).pdf
124.4 KB files/Food Preparation (13).pdf
676.37 KB files/Food Preparation (14).pdf
589.23 KB files/Food Preparation (15).pdf
223.94 KB files/Food Preparation (16).pdf
544.05 KB files/Food Preparation (17).pdf
1.48 MB files/Food Preparation (2).pdf
17.31 MB files/Food Preparation (21).pdf
1.19 MB files/Food Preparation (3).pdf
134.62 KB files/Food Preparation (4).pdf
593.95 KB files/Food Preparation (5).pdf
59.5 MB files/Food Preparation (6).pdf
189.67 KB files/Food Preparation (7).pdf
2.69 MB files/Food Preparation (8).pdf

❯ zimls zimgit-food-preparation_en_2022-03.zim |grep files
208.37 KB I/files/Food Preparation (1).pdf
2.21 MB I/files/Food Preparation (10).pdf
1.45 MB I/files/Food Preparation (11).pdf
3.17 MB I/files/Food Preparation (12).pdf
124.4 KB I/files/Food Preparation (13).pdf
676.37 KB I/files/Food Preparation (14).pdf
589.23 KB I/files/Food Preparation (15).pdf
223.94 KB I/files/Food Preparation (16).pdf
544.05 KB I/files/Food Preparation (17).pdf
1.48 MB I/files/Food Preparation (2).pdf
17.31 MB I/files/Food Preparation (21).pdf
1.19 MB I/files/Food Preparation (3).pdf
134.62 KB I/files/Food Preparation (4).pdf
593.95 KB I/files/Food Preparation (5).pdf
59.5 MB I/files/Food Preparation (6).pdf
189.67 KB I/files/Food Preparation (7).pdf
2.69 MB I/files/Food Preparation (8).pdf
114 MB I/files/Food Preparation (9).pdf

Looks like Food Preparation (9).pdf is missing. It is indeed missing from https://drive.farm.openzim.org/zimgit-food-preparation/zimgit-food-preparation.json which is the collection in https://farm.openzim.org/recipes/zimgit-food-preparation_en/config

[Popolechien]

But then that only partly explains the issue - what about the other files?

(post-disaster has the food-preparation files in it, so ok). but the -2MB variation in the other files is surprising

Moreover how can a file disappear from the JSON if said json hasn't been edited in a year?

[rgaudin]

Moreover how can a file disappear from the JSON if said json hasn't been edited in a year?

ZIM file is from 2022-03-15, last edit on 2022-03-22.

As for the 2MB difference, there's been numerous updates to nautilus since 1.0.1. Most important being not using zimwriterfs anymore. I am not investigating this unless there is a problem to solve.