Implement record timestamps source of truth

[pospi]

Following from https://github.com/valueflows/vf-graphql/pull/56#issuecomment-542520108 we need to think about how to handle timestamping in a trustless distributed system. This will be the logic that backs date_created and date_modified as per #40. This is different to the time the user says things were "created in the real world", for records which have such properties (for example, EconomicEvent.hasPointInTime).

I think this behavior will need to be configurable as resilience / trust levels depend on the network. There are a few options that could be taken:

• Trusted notary: good for trusted networks that are always available. In this case there would be a trusted autonomous agent listening on the DHT which registers timestamps when it receives new records. All other nodes use these timestamps for date_created and date_modified.
• Quorum time: good for untrusted networks with higher numbers of nodes. Configuration would be the number of nodes required to witness the record- each would write a timestamp, these timestamps would be averaged to get date_created & date_modified. Configured node count is also the minimum number of nodes that need to be online for the network to operate... if fewer nodes are online, the delayed observations will throw times out.
• Countersigned time: good for smaller untrusted networks. Basically like quorum time except that we only require one other agent to be online to witness a timestamp. Partners might be able to collude by syncing clocks.
• Trusted time: basically just taking people's word for it, assign date_created from whatever creation timestamp is provided (eg EconomicEvent.hasPointInTime).

[fosterlynn]

Some uneducated thoughts:

1. If possible, let's do something fairly simple now and code it so we can later plug in options if needed after we get some experience.
2. I don't think "trusted time" is a good idea, we're looking at a different meaning (time recorded in the system) - I think anyway.
3. Could there be another option of using the timestamp the initiator puts it in their source chain? It would then wander from there onto the dht, with that time recorded everywhere as the record creation time.

[fosterlynn]

I was looking at dcterms:created - and all of that vocabulary focuses on metadata. Maybe we should change our current use of dcterms:created to a vf term meaning when was it created in real life, an entered property. And use dcterms for our more metadata-y things referring to the digital resource (created, creator)?

[pospi]

Agree on doing the simple thing first... which for the moment is "trusted time". It's interesting that you say that you don't think that this method is a good idea, but are advocating for "timestamp the initiator puts it in their source chain"— those are effectively the same time, and have the same degree of "trustability" (i.e. both depend on the system time of the user writing the record).

[fosterlynn]

Agree on doing the simple thing first... which for the moment is "trusted time". It's interesting that you say that you don't think that this method is a good idea, but are advocating for "timestamp the initiator puts it in their source chain"— those are effectively the same time, and have the same degree of "trustability" (i.e. both depend on the system time of the user writing the record).

I understood that "trusted time" is entered by the user, because often vf:hasPointInTime will be entered by the user, not taken from system time. A lot of people can't or don't log their work on the spot.

[pospi]

Ah yes, I see. Separate thing that, record creation time vs time they say it happened. I guess it is up for the use-case to decide which fields are important.

On Fri, 1 Nov 2019 at 20:29, Lynn Foster notifications@github.com wrote:

Agree on doing the simple thing first... which for the moment is "trusted time". It's interesting that you say that you don't think that this method is a good idea, but are advocating for "timestamp the initiator puts it in their source chain"— those are effectively the same time, and have the same degree of "trustability" (i.e. both depend on the system time of the user writing the record).

I understood that "trusted time" is entered by the user, because often vf:hasPointInTime will be entered by the user, not taken from system time. A lot of people can't or don't log their work on the spot.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/holo-rea/holo-rea/issues/75?email_source=notifications&email_token=AADK2JBWMH6WI47MBDTQ6FLQRQAJNA5CNFSM4JGRKBCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEC2SBPI#issuecomment-548741309, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADK2JDQCAS6XXR4AEJIEALQRQAJNANCNFSM4JGRKBCA .

-- writings http://pospi.spadgos.com/ | twitter https://twitter.com/pospigos | github https://github.com/pospi ← where to find me ValueFlows https://www.valueflo.ws/ | HoloREA https://github.com/holo-rea/ ← things I'm working on

[pospi]

Further to this I wanted to outline what a "proper" implementation looks like to me given the current state of the emergent pattern that is Holochain apps.

I suspect this is best covered by a "mixin zome" that can be easily injected alongside other zomes in a DNA in order to provide the timestamping functionality as a pluggable service.

Open questions:

• The 4 flavours of timestamping outlined above (notary, quorum, countersign & trusted) might be best developed as entirely separate zomes. There is no reason for all functionality to be packaged together into 1 zome with configuration options, as this would simply make the zome WASM bundle bloated with unexecuted code. But- does the timestamping method used in a DNA potentially need to be changed dynamically without every user having to update the app? If so, the configurable option would be needed. It's also possible that a configurable version could be provided that simply composes methods from the other mixin zomes and wraps them in some config layer.

Suggestions & opinions:

• Zome API traits can be used to synchronise the public APIs of all mixin zomes.
• The default implementation of the timestamping zomes might as well include a public API for querying record timestamps. Implementors who do not want a public API exposed can import standalone entry & link type definitions into their projects without importing the API function handlers, as per https://github.com/holo-rea/holo-rea/issues/80.
• Using the same entry & link type names for every zome is the only requirement necessary to ensure that the same helper library can be used by implementors regardless of the actual zome used. In other words, this can give us polymorphism in the behaviour of timestamp links & entries.
• The helper library API should be a pretty simple thing, something like record_created_timestamp(Address) and query_created_timestamp(Address) et al. This provides simple read / write access to timestamping info from within third-party zomes in the same DNA; where the query_ logic and the means of calling record_ depend on the "timestamping flavour" being provided.

[fosterlynn]

For discussion, not making a proposal:

It seems to me that we should have one way of doing this, it is not an options-based kind of thing. It also seems then that we should just record all timestamps that could be useful so they are there; people can query and use them in whatever way they want, including averaging, etc. It also seems that there should be a fairly standard holochain way of doing this, just like there is de-facto in the database world - but maybe not, and then we can just roll our own.

Wondering, and possibly I am just way behind: Doesn't the key creation hashing imply that we can't add data to a record after it leaves the initiator? This would affect this issue and counter-signing. Are we pulling out this kind of data into new records related to the actual record?

[pospi]

It seems to me that we should have one way of doing this, it is not an options-based kind of thing.

Not possible, I'm afraid. Particularly when it comes to "quorum time", which has implications for the necessary online node count needed to maintain the network's timestamping functionality. A network with more participants might want a high number of distributed timestamps applied to prevent collusion. But a network with fewer participants would halt and become non-functional if it's expecting more timestamps than there are network participants available.

And then there are things like trusted enterprise environments where they want a central authority that handles timestamping (basically equivalent to R3 Corda's architecture) because they don't trust arbitrary nodes within their organisation to have clocks synced correctly.

Doesn't the key creation hashing imply that we can't add data to a record after it leaves the initiator?

Yes. The approach would just be to hang links off the main record with the timestamps stored in referenced entries. @philipbeadle has code he can provide from HoloFuel that does this- IIRC it requires manually pulling some fields out of the internal source chain in order to create a complete payload for signing.