search

markbates / pkger

Embed static files in Go binaries (replacement for gobuffalo/packr)
MIT License
1.19k stars 60 forks source link

`pkger` command generates differently on macOS Catalina vs Debian Buster #85

Open s12chung opened 4 years ago

s12chung commented 4 years ago

can we make these consistent so that we can commit the pkged.go files that don't change?

requaos commented 4 years ago

@s12chung are the binaries of the same version?

s12chung commented 4 years ago

double checked via go get-ing new binaries. on v0.15.1, macOS Catalina and Debian Buster generates different pkged.go files.

creker commented 4 years ago

Stumbled upon this myself. Wanted to write an automated check that runs CI and runs pkger to check that pkged.go doesn't change i.e. committed pkged.go is up to date. Turns out pkged.go changes depending on the environment. Checking its contents, ton of stuff is different - current module information, details of the included files (looks like OS specific information. Why even include such information?). Basically everything is different apart from the actual included files data.

maratori commented 4 years ago

My library only uses content of embedded files, and not metadata. So I came up with following workaround. Here is target for Makefile

embed-ui:
    @if [ ! -f ui/app/build-bs.md5 ] || [ -n "`find ui/app/build-bs -type f -print0 | xargs -0 md5sum | sort -k 2 | diff ui/app/build-bs.md5 -`" ] ; then \
        echo "Start embedding ui/app/build-bs into pkged.go..." && \
        pkger -o ui/server && \
        find ui/app/build-bs -type f -print0 | xargs -0 md5sum | sort -k 2 > ui/app/build-bs.md5 ; \
        echo "Embedding finished" ; \
    else \
        echo "No need to update pkged.go" ; \
    fi

Hash of embedded files are stored in ui/app/build-bs.md5. And pkger runs only if that file changed. This leads to reproducible builds. And I can check that all generated files are up-to-date on the CI.

creker commented 4 years ago

My workaround is kinda similar. My project is a tool written in Go. I implemented special command just for CI where I get a list of all embedded files, hash them and output md5. I run it first time with committed pkged.go and get md5. Then rebuild pkged.go, run it again and compare the results. If they're different I fail the build. With that I can be sure that I tag releases and accept pull requests with up to date pkged.go