search

melpa / package-build

Tools for assembling a package archive
https://github.com/melpa/melpa
25 stars 34 forks source link

Set file timestamps to reproducible value. #58

Closed houseofsuns closed 2 years ago

houseofsuns commented 2 years ago

The goal is to produce hash-identical archives whenever the source did not change. Previously timestamps caused this to fail as they are recorded in the tar archive. Now we set the timestamps of all files to the timestamp of the commit they are from.

This superseedes https://github.com/melpa/melpa/pull/8030 .

tarsius commented 2 years ago

The same can be achieved by passing the appropriate arguments to tar, see https://reproducible-builds.org/docs/archives/.

purcell commented 2 years ago

Yeah, I'd probably prefer to keep the special case handling out of this elisp if we can rely on tar instead.

houseofsuns commented 2 years ago

I'll investigate the tar options during the next week.

houseofsuns commented 2 years ago

I thought about dropping some of the reproducibility options, but then thought that it would leave things in a half-baked state which would only be more surprising if some problem arises.

tarsius commented 2 years ago

I have pushed the result to the repro branch.

I have ran into some other issues while looking at surrounding code and probably won't merge this into master until I have resolved those too. (And that might take a bit because of the nice whether.)

houseofsuns commented 2 years ago

Enjoy the sun. As long as this does not turn into the perfect being the enemy of the good everything is fine (maybe I'll prod a bit if nothing happens for a while). Also if there is something I could help with to speed this up, please tell (however I'll soon be offline until next week).

tarsius commented 2 years ago

Also if there is something I could help with to speed this up, please tell

What's the mercurial equivalent of git checkout (if branch (concat "origin/" branch) "origin/HEAD")? I haven't found anything with a quick search, but I also haven't tried asking variations of my question yet. (For git we don't bother updating local branches. That way we don't have to deal with rewritten history. We should probably do the same for hg.)

tarsius commented 2 years ago

I've merged the tarball bits.

houseofsuns commented 2 years ago

What's the mercurial equivalent of git checkout (if branch (concat "origin/" branch) "origin/HEAD")? I haven't found anything with a quick search, but I also haven't tried asking variations of my question yet. (For git we don't bother updating local branches. That way we don't have to deal with rewritten history. We should probably do the same for hg.)

Disclaimer: I'm not entirely sure about the semantics of HEAD if origin is a forge.

I would propose hg update (if branch branch "tip") as equivalent. "tip" points to the most recent commit which seems to be roughly what HEAD gives if no explicit branches are involved. I tested this locally and if history is rewritten hg update simply took the newest variant which is exactly the desired behaviour as I understand it.

houseofsuns commented 2 years ago

I've merged the tarball bits.

Cool. 🚀