search

mesg-foundation / js-sdk

Javascript mono-repo with all the tools to interact with MESG
https://mesg.com
4 stars 4 forks source link

Deterministic tar of system services #127

Closed NicolasMahe closed 4 years ago

NicolasMahe commented 5 years ago

I test once but the tar created from the same system source had a different IPFS hash using https://github.com/mesg-foundation/core/blob/dev/scripts/build-core.sh

@antho1404 suggested to use an option to not take into account the updated time when creating the tar.

antho1404 commented 5 years ago

I know in the js lib they have an option portable https://www.npmjs.com/package/tar, not sure about the go implementation of tar. This solves the problem, I had the same issue with the javascript CLI and solved it this way https://github.com/mesg-foundation/cli/blob/8490b4618aa498200a47599bc2a1132485caaa7a/src/commands/service/deploy.ts#L68

NicolasMahe commented 5 years ago

@mesg-foundation/core let's take a look for a quick solution, otherwise we will fix it when moving to CLI in JS.

NicolasMahe commented 5 years ago

actually, the js lib still keep the mtime:

portable Omit metadata that is system-specific: ctime, atime, uid, gid, uname, gname, dev, ino, and nlink. Note that mtime is still included, because this is necessary other time-based operations.
antho1404 commented 5 years ago

let's make sure that we find a way to actually reproduce it because I did some tests and my hash is consistent if I don't touch anything on the service so maybe that a particular use case. Let's not introduce a heavy development that will complexify it before we can reproduce it

NicolasMahe commented 5 years ago

Just to remind that this issue still exist. The mtime is taking into account when creating the archive.

The solution is to use git archive that remove all "metadata" and create an archive based only on the content of the files: https://git-scm.com/docs/git-archive The command looks like:

git archive --format=tar.gz --prefix=junk/ -o "/Users/nico/Downloads/dw2.tar.gz" HEAD

The main issue is a local git repo is needed for git archive to run..