Reproducible builds

[curiousdannii]

It would be a little bit nice if you could check out a release tag and get a reproducible build. The serial number would need to be hard coded. Possibly more stuff too? I'm not sure if it's possible or if there's some stuff I7 does that means it's not reproducible.

[angstsmurf]

I really don't know where to start with this.

I tried building the current code on Ubuntu and Windows, and there was a huge amount of differences in the .gblorb files, mostly in the blorb chunk headers and image chunks, oddly enough. I guess they may be using different versions of cBlorb.

Also lots of differences in the iFiction chunk, although the text looks identical. Different encoding, apparently?

Comparing only the ulx files, there are differences in six places, although I haven't counted the number of actual differing bytes. One difference is the header checksum, which makes sense, but the rest I have no idea about. It will take a better Glulx hacker than me to sort this out. Interestingly, many of the differences were only ±1.

Do you have any idea what may be causing this? Again, I suppose they might be using different versions of the Inform 6 compiler, and different versions of the ni binary as well.

[angstsmurf]

I suppose it might still be considered a reproducible build as long as you can make identical binaries on the same system. If so, it will be enough to make sure that the serial number is constant. That would have to be a script that changes it in the .inf file before the Inform6 compilation, or is there another way? Well, apart from changing the date on the build system.

[curiousdannii]

Thanks for looking into this. Even reproducible within one OS would be a good step. A shame that it's probably not possible between OSes. I started thinking about this because I had to rebuild the game in order to get a debug file, and thought it would be good if I could compile an exact replica of the official release. If the release process was documented (OS, post processing steps, etc?) that would be nice.

You can override the serial number with

Include (- Serial "123456"; -). 

I'd probably include it but commented out, and then uncomment it just before tagging the release.

[angstsmurf]

There isn't a lot of release process, really. I just build it with the official Inform 7 IDE on macOS. Then I insert the startup precomputation data into the blorb with the insert.sh script. That's it.

There was a problem with the old insert.sh script (which may point to some deeper bug), so I just now replaced it with a cut-down one that doesn't attempt to create a new precomputation data file, but just inserts an existing one.

I also committed you fixed-serial line. Feel free to make and push whatever changes (and new releases) you need.

[curiousdannii]

Ah, macOS means I won't be able to do it.

Does the macOS IDE give you an option to enable debug file output like the Windows one does (using the k switch)? And does it do so without turning on debug mode?

[angstsmurf]

As Andrew Plotkin mentions here, the Mac IDE will always use the k switch, so I think that means yes.

It does always generate a gameinfo.dbg when building. I don't think there is any way to switch this off in the IDE.

I compared some more compiler output, this time between Mac and Ubuntu. The Ubuntu ulx file header contains the text Info???6.330.38?, while the corresponding Mac header says Info???6.340.38? which sounds suspiciously like two different Inform 6 versions, and might explain some of the differences.

But I also compared the autogenerated .inf files, and here there are differences in the order of certain constants.

Mac:

[ E61 
    value ! Implied call parameter
    ;
    switch(value) {
        I71_german_language: print "German language";
        I72_italian_language: print "Italian language";
        I73_english_language: print "English language";
        I74_french_language: print "French language";
        I75_swedish_language: print "Swedish language";
        I76_spanish_language: print "Spanish language";
        default: print "<illegal natural language>";
    }
];

Ubuntu:

[ E61 
    value ! Implied call parameter
    ;
    switch(value) {
        I71_spanish_language: print "Spanish language";
        I72_english_language: print "English language";
        I73_french_language: print "French language";
        I74_swedish_language: print "Swedish language";
        I75_german_language: print "German language";
        I76_italian_language: print "Italian language";
        default: print "<illegal natural language>";
    }
];

This causes differences in many other places where these constants are used. Also the constants for first person plural and first person singular switches place between the versions. The rest is identical.

In theory, this means that you might be able to reproduce the official build by changing those in the .inf file.

Switch some constants around, replace Inform 6.33 with 6.34, re-write cBlorb to produce Mac-identical output on Ubuntu, and the build is reproducible.

[curiousdannii]

It does always generate a gameinfo.dbg when building.

Could you please upload it next time you do a release?

Switch some constants around, replace Inform 6.33 with 6.34, re-write cBlorb to produce Mac-identical output on Ubuntu, and the build is reproducible.

Sounds so simple when you put it like that ;)