search

tuna / tunasync-scripts

Custom scripts for mirror jobs
442 stars 170 forks source link

nix-channel GC: dangling narinfo files #136

Open linyinfeng opened 2 years ago

linyinfeng commented 2 years ago

Current GC logic deletes dead narinfo files along with the nar files they referring to. But live narinfo files can still refer to these deleted nar files, because multiple narinfo files may have same URL. https://github.com/tuna/tunasync-scripts/blob/a47eb031f58083f5a016c5090b233d22bd4b6b04/nix-channels.py#L429-L438

Narinfo file example:

# github:nixos/nixpkgs/e10da1c7f542515b609f8dfbcf788f3d85b14936#element-web
$ curl https://cache.nixos.org/hdz90ld1wwj6nwp580avv02v62cjh7h3.narinfo
StorePath: /nix/store/hdz90ld1wwj6nwp580avv02v62cjh7h3-element-web-1.10.10
URL: nar/0ji0p1g4fjwpqmf33maf7irznb2wclzi483hicyyjifybh43qxrs.nar.xz
Compression: xz
FileHash: sha256:0ji0p1g4fjwpqmf33maf7irznb2wclzi483hicyyjifybh43qxrs
FileSize: 13688456
NarHash: sha256:0r3pdgcnrlvf56cxlgxdsai5f1k9pf7cq80zssrbfbabrasbkk2v
NarSize: 43380024
References: 
Deriver: af669vfa956f7znxarma0rf1883nxy8k-element-web-1.10.10.drv
Sig: cache.nixos.org-1:T0b6vIGHgFC2E9SspFsf/MMubYixGGYna/JOssawEaQh4jdoyJNAkRNN9pJDFJDTPKER7DCXAVXE5zgzYedPBw==

# github:nixos/nixpkgs/c30945a93fbd3122a55ee6a63c9bfef7556bc82e#element-web
$ curl https://cache.nixos.org/zgwkj12lfii1ii041497bxm8rzcx23sd.narinfo
StorePath: /nix/store/zgwkj12lfii1ii041497bxm8rzcx23sd-element-web-1.10.10
URL: nar/0ji0p1g4fjwpqmf33maf7irznb2wclzi483hicyyjifybh43qxrs.nar.xz
Compression: xz
FileHash: sha256:0ji0p1g4fjwpqmf33maf7irznb2wclzi483hicyyjifybh43qxrs
FileSize: 13688456
NarHash: sha256:0r3pdgcnrlvf56cxlgxdsai5f1k9pf7cq80zssrbfbabrasbkk2v
NarSize: 43380024
References: 
Deriver: 51mif1gp7i3igq8d9a9aff073qn9drd4-element-web-1.10.10.drv
Sig: cache.nixos.org-1:kAT7/4P/GNm0kmFoTss6klYET4ZdYjNhs7OW3q3jDIQMXZqiTdXyZ8uLW0k2eiyuusgdBOSHzvZpcRjxWUFXBw==

Names of nar archives on https://cache.nixos.org are their base32 hashs. For example:

$ nix hash file <(curl https://cache.nixos.org/nar/1jh8kd7ql2v73fdspp1v08hvfdfxjvl2i86g409ckgpy8k6l41g8.nar.xz) | \
  xargs nix hash to-base32
1jh8kd7ql2v73fdspp1v08hvfdfxjvl2i86g409ckgpy8k6l41g8
dramforever commented 2 years ago

Confirmed. Practically though this seems to only occur really rarely and should not noticably impact usage.