search

petoju / terraform-provider-mysql

Terraform MySQL provider – unofficial fork
https://registry.terraform.io/providers/petoju/mysql
Mozilla Public License 2.0
63 stars 40 forks source link

How to reproduce the build #85

Closed xocasdashdash closed 10 months ago

xocasdashdash commented 11 months ago

Hi there! I'm trying to reproduce the build with the one that's published on the terraform website. This is part of how we onboard and lock providers from the community.

I've tried to run goreleaser locally but the hashes don't match, it might be an issue on my side so is there any instructions on how to generate the release artifacts? I want to generate the artifacts that are present in hashicorp's registry to download.

Specifically this one: 

curl  https://registry.terraform.io/v1/providers/petoju/mysql/3.0.37/download/darwin/arm64 | jq .
petoju commented 11 months ago

I am doing basically just this in directory of repo:

export GPG_FINGERPRINT=....
export GITHUB_TOKEN=...
goreleaser release --clean

My build machine is running these versions:

$ goreleaser -v
goreleaser version 1.16.1
commit: ba2c93bf2036905fe00fd8482f293febd58c1a81
built at: 2023-03-10T13:20:25Z
built by: goreleaser
goos: linux
goarch: amd64
module version: v1.16.1, checksum: h1:jOpwmxwyDtprQIGBPuDLRm5aVX/UaAaf894OCD+C3So=

https://goreleaser.com

$ go version
go version go1.20.3 linux/amd64

The version that is built is always tagged - the tag is v3.0.37 in this case.

If you have any suggestions about how to make it easier to verify or what can I provide, feel free to suggest it - ideally in form of PR. Bear in mind that out of abundance of caution, I don't store and don't want to store my signing key on any computer, so using any docker image is too complicated for me with too little benefit.

EDIT: I upgraded go binary at some point. You can always find the binary used using "strings" on the binary itself. I don't use any special patches to my go binaries, only whatever Ubuntu provides.

xocasdashdash commented 11 months ago

Appreciate the answer! I think having a script in the repo with the publishing actions could be enough.

I'm still not able to fully repro your changes, but it might be related to the fact that i use darwin and not linux.

If i clone the repo and do the following:

git checkout v3.0.37
goreleaser release  --skip-publish --skip-sign  --clean -p 2

I get the following shassums:

09e8be2e8841cc8990c1d0f3af1c9fe5dfde2dba7e834469220386cc5457eefa  terraform-provider-mysql_3.0.37_linux_386.zip
14489da827d2b828dfb3f819ccd9d33e9fc71ea88c7c5c63d8f983c23fba2be0  terraform-provider-mysql_3.0.37_linux_amd64.zip
31be9f40599544e529aca71d0903bd819fd02410ee828823a3d567d90e64fdce  terraform-provider-mysql_3.0.37_freebsd_arm.zip
5f24375e180c5f11cb0accf29d82fd145d3dcf8ea2b4aab18b225870bf8ba755  terraform-provider-mysql_3.0.37_freebsd_arm64.zip
65f3f7548554652a53c86cf8b1e31897e4682973706fc87d39b00e3006af7664  terraform-provider-mysql_3.0.37_windows_arm64.zip
8429f899140d62f9f7dd9fafd524f82bfc61cb8c241ae7d37ff63d35b6857a6a  terraform-provider-mysql_3.0.37_linux_arm64.zip
852eb9b98467fa705c0a71ea62521bf4c0e61e1fd79902e194f9db681f5ddb2e  terraform-provider-mysql_3.0.37_freebsd_amd64.zip
c1a965e0447753654dda9e09035816a6b75e67186d295983a55d11e5e6abccc9  terraform-provider-mysql_3.0.37_windows_arm.zip
c34ecc249723e99a83459c17346a6ae821fcc2b737a931a77d336812a5bcae8c  terraform-provider-mysql_3.0.37_darwin_amd64.zip
c855c615ec2c27bc1af52f6bb7b3661b5060ff2eb820a99e1654e3103f584a68  terraform-provider-mysql_3.0.37_linux_arm.zip
cd163f5296d576aa84beeb389984ca585d249f9694ac0e1f9aca16179a771008  terraform-provider-mysql_3.0.37_windows_amd64.zip
ec2327fccab98a6a79e75d956ba2afc7d6e801b12516d3d8d0845a669bf21a82  terraform-provider-mysql_3.0.37_windows_386.zip
f6af85b81b7dc4d1d2a53e0d7748ed5d3a88b13fba0bbfed541ffa767581bead  terraform-provider-mysql_3.0.37_darwin_arm64.zip
fe9ad4a9a3a536da18a647c0f09581965c9de1160ce6e44a045f16cfbbb4040a  terraform-provider-mysql_3.0.37_freebsd_386.zip

This matches the same value of running:

shasum -a 256 dist/terraform-provider-mysql_3.0.37_darwin_arm64.zip

But not the value of doing:

curl -L -O https://github.com/petoju/terraform-provider-mysql/releases/download/v3.0.37/terraform-provider-mysql_3.0.37_darwin_arm64.zip
shasum -a 256 terraform-provider-mysql_3.0.37_darwin_arm64.zip
petoju commented 11 months ago

@xocasdashdash I did some experiments and it looks like it is difficult to impossible to reproduce current binaries.

I tried these things, that affected checksums:

These did not affect checksums in my experiments:

Then I searched a bit and based on https://github.com/golang/go/issues/57120 it seems that Go 1.21 should have reproducible builds. But as it was released only yesterday, I don't plan to move there immediately. That said, it wasn't mentioned in changelog, so we don't really know.

EDIT: One more suggestion, that will probably make it work (without guarantees).

ZIP files contain checksums and ordering and other randomness. Unpacking removes this part.

I did this for some windows arm64 binaries (to be sure my system binaries are not used) and data match after unpacking. That provides proof that the build was executed on the current sources.

xocasdashdash commented 10 months ago

mmm interesting, i did that myself with the darwin_arm64 binary and it's failing. I might try to go with a linux container so i can do the linux build and see if they match

petoju commented 10 months ago

@xocasdashdash with v3.0.39 I started using Go 1.21.0 and it seems to be easily reproducible using different OS and different releaser. I use the same Go version, though.

Make sure you use Go 1.21 and then, you can try verifying it.

$ brew install go@1.21
$ brew install goreleaser

For the record, shasums of unzipped files are:

$ cd dist
$ shasum -a 256 terraform-provider-mysql_*/*
a57fef503732f75ce5359f33695a0d008eb5f33e308779f1e8c037a41962c746  terraform-provider-mysql_darwin_amd64_v1/terraform-provider-mysql_v3.0.39
549211d6ee5fcb7f2d0d8b64b90a8beec168384354407f33f3868b2da06f24d1  terraform-provider-mysql_darwin_arm64/terraform-provider-mysql_v3.0.39
47aef734be3fe6744a073689438a352b73693830a34cb0bdfda1e7485a798388  terraform-provider-mysql_freebsd_386/terraform-provider-mysql_v3.0.39
ce80e5192b837930b19493f7646be92c3ea96721b972330b949f55bf76be0f36  terraform-provider-mysql_freebsd_amd64_v1/terraform-provider-mysql_v3.0.39
f8a56c00dec87a69c42526abb806199cd4a43cda250c043ee9b86bd69204b8b5  terraform-provider-mysql_freebsd_arm64/terraform-provider-mysql_v3.0.39
c476a29b70fd6fddae7ef965160cb8ee99732bddd7b04d2272be82e241290b43  terraform-provider-mysql_freebsd_arm_6/terraform-provider-mysql_v3.0.39
6e19c5a01264e122081b8e65c4ee95aa2500b92bc440a01bebbd40e969ebe48d  terraform-provider-mysql_linux_386/terraform-provider-mysql_v3.0.39
6c5d66f6a75671247c5dbecf8422ded7a1e8cb1fefadc42234a24f875c297665  terraform-provider-mysql_linux_amd64_v1/terraform-provider-mysql_v3.0.39
2cab6f0e562275f7a4a076ff08bf59b265dccadcecda213e053c1b5007be83eb  terraform-provider-mysql_linux_arm64/terraform-provider-mysql_v3.0.39
f8a18eb6510ce68cf32fe6779a5c5e9023651f47dbf8396f63b52f60ac3f41d9  terraform-provider-mysql_linux_arm_6/terraform-provider-mysql_v3.0.39
d2b72bf64fc9f67a90bff31ceb12415a14f1c38c610948304c67a2b049ba3689  terraform-provider-mysql_windows_386/terraform-provider-mysql_v3.0.39.exe
1a2aff5a98039a3323ca03cda221bbd8305051eaba4b21bbc58e7a0c8f1c80e4  terraform-provider-mysql_windows_amd64_v1/terraform-provider-mysql_v3.0.39.exe
c8515bcf70591f5eb04108b22c408047b5bf5946d0edb64495a19de672d26bd2  terraform-provider-mysql_windows_arm64/terraform-provider-mysql_v3.0.39.exe
d32bcc4f882375a4261bcf045439e7581269ded1151c86f16176cb170b665185  terraform-provider-mysql_windows_arm_6/terraform-provider-mysql_v3.0.39.exe
xocasdashdash commented 10 months ago

Hey! Thanks so much for doing this!

I'll try it out! :)

xocasdashdash commented 10 months ago

I can confirm that it works with the binaries, the zip files are not, but that's fine :D