Are there plans to integrate with this spec: https://chainpoint.org/ ?

I've used OTS with success and now am working on some solutions on ethereum. It seems the chainpoint spec is flexible to accommodate both. Would you be interested in a converter?

OpenTimestamps is more flexible than the current Chainpoint spec, so it's not possible to convert OpenTimestamps proofs to the Chainpoint format. However, all Chainpoint proofs can be converted to OpenTimestamps proofs.

And yes, I'm sure some people will find it useful to be able to upgrade their Chainpoint proofs to OpenTimestamps!

Perhaps I can add a little color to this as one of the authors of the upcoming Chainpoint v3 proof standard. Our current proof format v2 is perhaps what @petertodd is referring to and I would be curious to hear his specific concerns. We have not been standing still though and we are nearing formal release of version 3 of the standard which we are documenting here.

https://chainpoint.org/

I think you'll find that v3 is even more flexible, developer friendly, and fully documented (including schema validators) in both its JSON-LD and binary forms. I've found it hard to judge the claimed 'flexibility' of the OTS format since its binary only implementation spec and design goals are largely undocumented (although there are a couple of implementations in code).

We are always looking to enhance the capabilities and flexibility of our proof format (which is on the W3C standards track), which is why we have created v3. The Chainpoint v3 proof format, documentation and reference implementations will be released shortly as one component of a broader vision we have for Chainpoint.

We'd be happy to see the community develop adapters or client code that can read/write/verify Chainpoint or OTS proofs interchangeably. We'd also encourage the OTS project to consider adopting this open format, or contributing enhancements or filing issues to address any perceived shortcomings. We believe the broader timestamp community would benefit from all players using a common standard, and we believe we offer the best, and most broadly accepted, option to do so.

Cheers.

Our current proof format v2 is perhaps what @petertodd is referring to and I would be curious to hear his specific concerns.

Chainpoint v2 is obviously incapable of representing most OpenTimestamps proofs due to the fact that the v2 format lacks commitment operations. If you think otherwise, please generate a set of Chainpoint proofs matching the examples in https://github.com/opentimestamps/opentimestamps-client/tree/master/examples

It's frankly dishonest of you to imply otherwise. Put another way, if the v2 format didn't have such obvious issues, what was the rational for developing the v3 format?

I think you'll find that v3 is even more flexible, developer friendly, and fully documented (including schema validators) in both its JSON-LD and binary forms.

A developer friendly format in the cryptography and consensus space is one that maximizes the chance of an implementation returning correct results, even in the face of adversaries. Making it easy to write implementations that appear to work correctly, but don't actually work correctly under all circumstances, is very developer unfriendly.

Quite a bit of effort has gone into making OpenTimestamp's binary format developer friendly. It is very difficult to parse incorrectly, and the resource usage of correct parsers is bounded by obvious constraints. It's not quite perfect - there's active discussion right now among the devs about a possible improvement to round-trip use-cases, and there's one more improvement to make to its forward-compatibility story - but it's very close.

Remember that when you use things like JSON-LD and MessagePack, you end up with not one parser but two parsers: one parser for the lower-level encoding, and a second parser to convert the tokens emitted by that lower-level encoding to the actual in-memory representation. This duplication of effort is a major source of exploits.

I've found it hard to judge the claimed 'flexibility' of the OTS format since its binary only implementation spec and design goals are largely undocumented (although there are a couple of implementations in code).

OpenTimestamps is specified by documented code rather than English text because the history of defining consensus systems with English text is terrible. Time and time again consensus systems alleged to be specified by English-language specifications have had their implementations diverge from those specifications, with the response being that the English-language specification was modified to match the implementation. In cases where there are multiple commonly used implementations, often the implementations have both disagreed with each other and the English language specification (as happened to Ethereum recently).

Or put another way, the OpenTimestamps specification is a set of math equations with commentary written in English text. Now I'm sure we can improve on the way those math equations are specified, and improve the commentary on them, but time and time again past experience has shown this is a highly effective way - probably the most effective way - to write specifications for consensus systems.

We'd be happy to see the community develop adapters or client code that can read/write/verify Chainpoint or OTS proofs interchangeably.

Given that the Chainpoint v3 specification is nearly - but not quite - an identical copy of the OpenTimestamps specification - yet no-one is using it yet - the most productive thing you could do is simply adopt OpenTimestamps so we end up with one specification rather than two. For those who must use JSON-LD, we can always define a 1-to-1 mapping of the canonical binary format to JSON-LD.

After all, OpenTimestamps is already widely used; Chainpoint v3 isn't. I see no rational other than "not-invented-here" for creating yet another specification when one already exists.

If you would like to go down this route, I'm open to working with Chainpoint in upgrading existing Chainpoint v1 and v2 proofs to the OpenTimestamps specification.

Peter, the Ad hominem attack on me personally was not really necessary was it? It comes across as more than a little defensive. I'll try and address some of your points though.

A developer friendly format in the cryptography and consensus space is one that maximizes the chance of an implementation returning correct results, even in the face of adversaries. Making it easy to write implementations that appear to work correctly, but don't actually work correctly under all circumstances, is very developer unfriendly.

You seem to be implying there is a flaw in our approach towards correctness, and yet you don't specify one. Sounds like FUD. We believe our approach to clear specification, combined with well documented and commented reference implementations, schema validation and other aspects yet to be announced serve our community of users and developers well.

Remember that when you use things like JSON-LD and MessagePack, you end up with not one parser but two parsers: one parser for the lower-level encoding, and a second parser to convert the tokens emitted by that lower-level encoding to the actual in-memory representation. This duplication of effort is a major source of exploits.

And now you imply that we are unsafe (FUD) while once again providing no basis for your claim. I would be curious to hear your ideas on the threat model you feel that users of a Chainpoint or OTS proof are up against. I for one am comfortable with our choice, from a security standpoint, of battle tested, and pretty utilitarian, technologies such as JSON, MessagePack, and zlib. Our complete binary serialization/deserialization library is about 60 lines of code as a result of the use of these common encodings. There is little, if anything, about the choice of these technologies that open us up to exploits, and developers get a format that they can trivially pack/unpack in 50+ language runtimes, or in browser. Proof files are merely containers for hashes and a set of simple instructions for combining those hashes. The proof file format itself offers little to nothing in the way of security guarantees, it is the strength of cryptographic hash functions that provides all of the security. No container format choice will change that, and you know that well.

Given that the Chainpoint v3 specification is nearly - but not quite - an identical copy of the OpenTimestamps specification - yet no-one is using it yet - the most productive thing you could do is simply adopt OpenTimestamps so we end up with one specification rather than two.

We have thousands of users, developers, corporations, and even competing timestamp services using or expressing interest in Chainpoint today. They will find a clean and simple migration path forward from Chainpoint 2.0 to 3.0.

After all, OpenTimestamps is already widely used; Chainpoint v3 isn't. I see no rational other than "not-invented-here" for creating yet another specification when one already exists.

Your "not-invented-here" mention is an interesting viewpoint for you to take considering OTS was first announced on September 15th, 2016 which was exactly:

one day after W3C Chainpoint Community Group was announced
One month after Chainpoint 2.0 was announced
Fifteen months after the original Chainpoint 1.0 specification was released.

Full credit to you for your early implementation of a timestamp service that you rewrote and turned into OTS, but the world didn't stand still while you decided what to do with it.

Apologies to @zmitton for the extra drama in his well-intentioned feature request. I hope your question might have been answered in the process though.

Cheers

Peter, the Ad hominem attack on me personally was not really necessary was it?

You were being dishonest; that's not an ad hominem attack, that's me calling out you being dishonest.

You seem to be implying there is a flaw in our approach towards correctness, and yet you don't specify one. Sounds like FUD.

Sorry if I wasn't sufficiently clear: your approach towards specifying a cryptographic protocol with English text, with the protocol built upon non-consensus-designed serialization layers has failed repeatedly in the past. Note how I both gave examples of those failures, and an explanation of why they happen.

That's not FUD, that's education.

And now you imply that we are unsafe (FUD) while once again providing no basis for your claim.

Ah, you seem to be thinking that issues like memory safety are the difficult part here - they're not. Modern languages have made memory unsafety a thing of the past in competent designs. What we're up against is failure to reach consensus: it's a failure if two OpenTimestamps implementations disagree on the validity of a proof. Not as stop-the-world severe as this problem is in Bitcoin, but still a major failure, and in any case, a design goal of mine is for OpenTimestamps to be usable in consensus systems.

To say that "There is little, if anything, about the choice of [JSON-LD and MessagePack] that open us up to exploits" and "The proof file format itself offers little to nothing in the way of security guarantees" deeply underestimates the problems. For example, DoS attacks are absolutely a security problem, and good file format design is required to prevent them.

We have thousands of users, developers, corporations, and even competing timestamp services using or expressing interest in Chainpoint today. They will find a clean and simple migration path forward from Chainpoint 2.0 to 3.0.

You haven't answered my question.

Why is Chainpoint 3.0 gratuitously incompatible with OpenTimestamps, given that Chainpoint 3.0 standard is almost, but not quite, an exact copy of OpenTimestamps? Migrating those users to OpenTimestamps would be every bit as clean and simple, and has the advantage of not unnecessarily creating yet another timestamping format. After all, Chainpoint 1.0 and 2.0 are radically different standards than 3.0

Your "not-invented-here" mention is an interesting viewpoint for you to take considering OTS was first announced on September 15th, 2016 which was exactly:

Why are you bringing up this irrelevant and misleading ad-hominem? I could easily rebut it, but lets stick to the tech: Why is Chainpoint v3.0 being created, when it's almost, but not quite, exactly the same thing as OpenTimestamps? What is your rational for further fragmenting the ecosystem?

Thanks for the "education" Peter. I'll reply here instead of to your cherry-picking quotes from my responses on Twitter. Reply if you wish, but this conversation has run its course for me.

Sorry if I wasn't sufficiently clear: your approach towards specifying a cryptographic protocol with English text, with the protocol built upon non-consensus-designed serialization layers has failed repeatedly in the past.

You consistently imply that providing our developers, users, and business partners with clear descriptions of our proof format is a flaw, or even a weakness. That somehow enforcing proof spec compliance with a valid schema is not relevant. That code is the only thing that counts. I take a larger view, code absolutely counts, so does the ability to learn and reason about that functionality by non-developers. The two are not mutually exclusive. Both are important.

What we're up against is failure to reach consensus: it's a failure if two OpenTimestamps implementations disagree on the validity of a proof. Not as stop-the-world severe as this problem is in Bitcoin, but still a major failure, and in any case, a design goal of mine is for OpenTimestamps to be usable in consensus systems.

I agree completely. We have the same design goals for Chainpoint. The on disk/wire format of the container is orthogonal to this though in my opinion. I'd like to address this further after the upcoming Consensus conference.

For example, DoS attacks are absolutely a security problem, and good file format design is required to prevent them.

The design of the file format will never fully prevent or mitigate actual DoS attacks against validating HTTP services. The servers receiving proofs to verify must of course do sanity checking on incoming hashes or an incoming proof. Our systems will continue to do validation of these the same as I assume your systems do and reject those that are not well-formed and valid. If clients are sending malicious proofs to either system en masse I don't see how the OTS binary file format can claim some immunity from that type of attack.

Why is Chainpoint 3.0 gratuitously incompatible with OpenTimestamps ... Migrating those users to OpenTimestamps would be every bit as clean and simple, and has the advantage of not unnecessarily creating yet another timestamping format.

We are incompatible because Chainpoint proofs have existed in various versions that pre-date OTS as I pointed out earlier and it has of course evolved. Your use of the word "gratuitously" implies we are creating new versions of our standard to spite you personally, or perhaps to spite what you consider to be a technically superior solution. Nothing could be further from the truth. Note that I have never denigrated you personally or the impressive OTS solution you have created. The only beef I have with OTS is that the format is largely opaque, making it not very suitable as a 'standard' in the conventional sense IMHO. Chainpoint 3.0 is a result of a design effort that took place when I joined Tierion/Chainpoint and which took into account the multi-year experience of our founders running a commercial timestamp service, and shepherding a new standard, and my own experience developing large scale systems that operate at scale.

We did choose to create an evolution of that format that suits our design goals and meets the needs of our customers and the broader community. By your own proclamation "OpenTimestamps is alpha software" and is commercially untested. You are unable to point us to documentation or design criteria that explain its benefits other than a single blog post. Why would we make that choice? How would we justify it to the investors in our business that support us in the effort to pursue an open standard for the mutual benefit of many?

What is your rational for further fragmenting the ecosystem?

I could ask you the same question I suppose.

Hi @zmitton OpenTimestamps have experimental support for Ethereum. Here you can find a .ots receipt cross timestamped on bitcoin and ethereum, while here you can see it with an online visualizer. There is also an experimental running eth calendar that you can try at https://eth.ots.eternitywall.com/ (do not use this for production cause the url it's not final) the code for the server is at https://github.com/RCasatta/opentimestamps-eth-server Standard clients will print info of the receipt but they still are not able to verify ethereum attestation. If you want to verify the ethereum attestation you must use the keccak branch of this repo

do not use this for production cause the url it's not final

Note that while we havne't actually added this to the software, it's pretty easy to fix URL changes in timestamps after the fact in most situations. Even if you can't change the timestamp itself (IE w/ timestamped git commits) we can always add a uesr-configurable remapping table. And of course, if you do use ETH timestamping, you can and should simultaneously timestamp in the Bitcoin calendars as well!

@RCasatta Looks like https://eth.ots.eternitywall.com/calendar/ isn't setup to enable downloading the Eth calendar like the other calendars; might be an idea to do that. Another good idea might be to add a word like "alpha", "test", "temporary", etc. to the URL to make it 100% clear that we're not making any guarantees quite yet.

You consistently imply that providing our developers, users, and business partners with clear descriptions of our proof format is a flaw, or even a weakness. That somehow enforcing proof spec compliance with a valid schema is not relevant. That code is the only thing that counts. I take a larger view, code absolutely counts, so does the ability to learn and reason about that functionality by non-developers. The two are not mutually exclusive. Both are important.

Neither code nor a English language spec written with the aim of achieving consensus are readable by non-developers; Ethereum's failed attempts at achieving the latter are a case in point:

Ethereum Yellow Paper Extract

Another example is the notoriously unreadable C++ specification, which very, very few programmers actually use directly, let alone non-developers.

What those audiences need isn't a specification of any kind, but rather documentation. For non-developers in particular, I think my announcement blog post is exactly the kind of documentation they need, describing what timestamping can and can-not do, the theory behind how a cryptographic timestamp works, and finally, how the OpenTimestamps commitment operations and calendars work.

For developers who want to implement OpenTimestamps in their applications, we could use both better API documentation, and more OpenTimestamps libraries in addition to our Python, Java, Javascript, and Rust libraries. But again, for that audience English language specifications are misleading, even dangerous: if you can't read the Python code you probably shouldn't be reimplementing a cryptographic protocol like OpenTimestamps, unless of course you're doing so for educational reasons.

The design of the file format will never fully prevent or mitigate actual DoS attacks against validating HTTP services. The servers receiving proofs to verify must of course do sanity checking on incoming hashes or an incoming proof. Our systems will continue to do validation of these the same as I assume your systems do and reject those that are not well-formed and valid. If clients are sending malicious proofs to either system en masse I don't see how the OTS binary file format can claim some immunity from that type of attack.

Sorry, but you are very confused here. You should read up on the LANGSEC design principles.

One very simple example of this in action is how OpenTimestamps proofs have explicit length limits on opcodes to ensure that even the most constrained implementations can predict in advance the maximum RAM requirements to deserialize and validate a timestamp. A more subtle example is in the way that OpenTimestamps proofs can be deserialized and validated without maintaining any context other than the (length-limited) messages at each fork point.

Details like this are essential to ensuring that attackers are unable to exploit performance/resource issues during DoS attacks; my goal with OpenTimestamps is to ensure even highly constrained stackless embedded implementations can parse and validate proofs efficiently.

We are incompatible because Chainpoint proofs have existed in various versions that pre-date OTS as I pointed out earlier and it has of course evolved. Your use of the word "gratuitously" implies we are creating new versions of our standard to spite you personally, or perhaps to spite what you consider to be a technically superior solution.

That's missing the point entirely. As we both know, Chainpoint v1 and v2 simply can't represent all timestamps that OpenTimestamps (and the almost identical Chainpoint v3) formats can. Equally, v2 and v3 proofs can always be converted into OpenTimestamps proofs.

Equally, I'm sure we both agree that commitment-operations-based timestamps are a good upgrade to prior less flexible merkle-tree systems like Chainpoint v1/v2.

Thus, the only question we have here is when v1/v2 users upgrade their systems, what should they upgrade too? Given that OpenTimestamps already exists, my question is what are the technical reasons for developing the almost identical Chainpoint v3 standard?

Nothing could be further from the truth. Note that I have never denigrated you personally or the impressive OTS solution you have created. The only beef I have with OTS is that the format is largely opaque, making it not very suitable as a 'standard' in the conventional sense IMHO. Chainpoint 3.0 is a result of a design effort that took place when I joined Tierion/Chainpoint and which took into account the multi-year experience of our founders running a commercial timestamp service, and shepherding a new standard, and my own experience developing large scale systems that operate at scale.

Could you explain more what you mean here by "The only beef I have with OTS is that the format is largely opaque, making it not very suitable as a 'standard' in the conventional sense IMHO."?

I could ask you the same question I suppose.

Again, OpenTimestamps predates Chainpoint and was developed well before Chainpoint was. How can I be fragmenting the standards ecosystem if at every step in the development of OpenTimestamps, no comparable timestamp proof format even existed?

@petertodd I setup the calendar backup at https://eth.ots.eternitywall.com/calendar/ I also added a text warning that is an experimental feature in the home page of the calendar, at the moment I am not changing the url since it's not used from any of our libraries (people must explictely pick the url in this issues where is clear that is an experimental calendar)

opentimestamps / opentimestamps-client

Are there plans to integrate with this spec: https://chainpoint.org/ ? #43