typeable / nixpkgs-stackage

Stackage overlay for Nixpkgs
25 stars 13 forks source link

WIP Make all-cabal-hashes derivation reproducible #26

Closed binarin closed 5 years ago

binarin commented 6 years ago

Part of https://github.com/typeable/stackage2nix/issues/41

Initially all-cabal-hashes derivation included a .git directory, which were used to fetch a .cabal file content given a sha1 hash. The problem is that there is no canonical representation for .git directory and every clone is different.

This patch gets rid of .git directory completely. The sha1->content mapping that was contained here is instead explicitly materialized on top of a filesystem.

Old derivation already contained several hundred thousands of files, with this patch this amount approximately trippled. It's not a significant degradation, but still something that needs to be fixed.

Initially I though about using git .pack/.pack.idx format to store the mapping, but sadly it won't work for storing mapping from package name/version to a .cabal-file. Something should be done about this (put everything into .zip-file? find some other suitable format? come with a format of our own?). But for now this implementation should be tolerable (file count "barely" grows from 200k to 500k).