Question: deterministic serialization

[nolash]

First of all, thanks for an awesome project.

I've implemented a setup where every update adds a second update with a signature on the previous.

I understand that the interpreted data is deterministic, it terms of the values set. But on dicts, is the serialization actually deterministic too? As far as I know, json intepreters do not have to enforce any order in dicts. In my case it's imperative that:

• Alice makes an update
• Alice signs
• Alice does Automerge.save()
• Bob does Automerge.load()
• Bob loops through history
• Bob creates a "digest" with Automerge.save() on even index numbers
• Bob calculates a signature over the "digest" on odd index numbers

Can this be guaranteed to work, no matter js engine etc?

[ept]

Hi @nolash, the result of Automerge.save() is not deterministic, so the exact procedure you describe won't work. However, it is possible to make signatures work like this:

• For every update that is made, use Automerge.getChanges() to get the change, and encode it to JSON using JSON.stringify
• Sign that JSON string and store the JSON string along with the signature
• Don't use Automerge.save(), but just rely on those stored strings. Instead of Automerge.load(), you can load a document by taking all the strings, parsing them, and passing them to Automerge.applyChanges()
• Those strings and signatures are sent verbatim to collaborators, who check the signatures before parsing the JSON

Even though JSON does not guarantee any particular order of field names, this should work, since the signature is on the encoded string. Will this do what you need?

[nolash]

Yeah it will, it does bloat the graph, however.

What would be cool was if the save function could modified (or a hook be added) to indeed guarantee eg. alphabetical dict keys in serialization.

... or of course can do it in the application, too, by loading the save serialization and re-serialize in order. What I'm after needs only to be application specific - as in the alphabetical ordering can be part of the protocol.

[ept]

The binary encoding format being developed on #253 is designed with deterministic serialisation in mind. At the moment its implementation is not yet fully deterministic, but it's working towards that direction. Hopefully this will do what you need in the future.