upgrade to tar 1.29

[mark-kubacki]

We will need the following options to tar for reproducible builds. Therefore this PR upgrades the version dgr ships with to 1.29, but still calls the host's tar by default.

• --sort
• --clamp-mtime
• --exclude

The first two commits prepare the CI and the required version of tar. Any later commits progressively use more of the new features.

Commits are – following the industry best practices – split into logical units. That is, every commit motivates its own use-case, and/or can be reverted independently later.

Please note that Github (the web frontend) could display the commits in the wrong order.

[n0rad]

can you squash please

[n0rad]

Don't add tar binary to repository.

Please remove it and add it as ignored until we update buildroot

[n0rad]

squash, change the log stuff and remove binary from repo and it's good to me

[mark-kubacki]

I wish we could do this entirely without resorting to the tar binary. But all implementations in Go are by far not on its level feature-wise. And indeed, won't be in the next months: GNU tools can and use Perl regular expressions, Go's implementation a reduced set by design. So even if works on a Go implementation started today (which sounds trivial), that'll be a complex roadblock for whoever fails to plan for this.

An upgraded tar in the buildroot won't do it: We already need to have a defined tar on the host to build dgr. Except perhaps you want us to extract it from the buildroot as part of gomake (a name clash with the tool gomake, by the way) after cleaning or the like. Anyway, as-is there's no way around building tar and/or shipping a prepared one. The latter approach is easier on the requirements (no gcc needed).

I will squash this into three commits:

1. Update the CI, as it is independent of the others.
2. Above tar upgrade, which we revisit the second or third time.
3. Anything else related to Go source code, »utilize the new options«.

[n0rad]

it's true that it would be better with a go version but here I'm just saying that I don't want to add more binaries to the git repository and that we should build it instead. buildroot tar is in repo because it's very long to build (about 1h) but I don't like it too. Tar build should be fast enough that we can build it once per dev checkout.

Also like I sayed, it's a temporary fix and will be included in the buildroot later.

[mark-kubacki]

Addressed.

Most of the time went into pleasing Travis.

[n0rad]

I'm looking at your PR to try to merge it but I see you are mixing 2 things. Building dgr as reproductible and building aci as reproductible.

I merged the first PR that is using latest version of tar to build reproductible aci during the build. I did not included the rest for the moment since it's building tar to use it to build dgr.

on another PR @nyodas updated buildroot with latest version of tar so you should be able to go ahead with building reproductible aci. Also a test that is building an aci twice to compare diff is welcome if you want your reproductible build not to be broken by further changes.

Now about building dgr as reproductible, instead of downloading tar and using it, we should build dgr inside a container to solve this, like @nyodas did to do buildroot.