Need of a test suite to achieve interoperability

[FabioPinheiro]

Interoperability refers to the ability of different systems or software to work together seamlessly. In order to achieve interoperability, it is necessary to ensure that each system or software component is capable of communicating with the others in a standardized way. One of the most effective ways to achieve this standardization is through the use of a test suite.

A test suite is a collection of tests designed to ensure that a system or software component meets a set of requirements or specifications. These tests can be used to verify that the system or software component is capable of communicating with other systems or components in the expected way.

I want to create my own implementation of DIDComm v2. To be type-safe and idiomatic for functional programming in Scala.

• So I did an implementation
• I tested against all of the test vectors from the spec. Which was hard but really satisfying when I was able to read messages from other people not only my own messages.
• Then I found problems when I was reading messages from other JVM implementations of DidComm. Basically, the problem was that the Base64 encoding of the character '\' was done in a legacy mode. My implementation did not support that at the time. It doesn't matter and I also don't expect the test Suite 2 to cover everything.
• Now I'm trying to have simple execution of the trust ping protocol with the roots-id/didcomm-mediator. I had to dig into the code of their dependencies to find what the problem seems to be. https://github.com/sicpa-dlab/didcomm-python/pull/93

I'm assuming that they are not following the specification. But the point is I shouldn't have to look at and understand the code of other implementations. I should be able to test against a test suite.

How can we be talking about having interoperability between agents if we don't even test (or have a good way to test) interoperability at the DID Comm library level. Seems even a bit ironic.

Considering DID Comm specs a meta protocol this is even before starting to talk about the protocols. I don't know which libraries implement which version of DID Comm, what extensions they support, etc.

Don't take me wrong. I pretty much still believe in DID Comm. Otherwise, I wouldn't have spent most of my free time from the last 12 months. Doing an implementation of DID Comm for Scala/ScalaJS. If I didn't believe how well ScalaJs fits the development of Dapp for web3 and how communication is a crucial part of it.

They are also other problems that need to be addressed:

• There is a clear lacking of tools to develop and debug agents. Which makes the creation and adoption of any advanced protocol almost impossible.
• There seems to be redundant data on the encrypted message and the content inside. (like field 'to' 'from' etc)
  • Which concerns me on the security level. Have the implementations the necessary checks?
  • Also concerns me about computer resources special because it is redundant. (On a simple message the 'to' and 'from' can easily count for three-quarters of the data...)
• At the current state of DIDComm specs and current protocols. I believe DID Comm wouldn't work if someone took down the DNS. Which defeats the whole point of being decentralized... Feel free to point me wrong.
• It's also crucial that Did Comm gains attraction before being obsolete. I mean creating stuff for the common user:
  • Chat apps
  • Simple games
  • Useful tools like https://www.random.org/ (How many have used this when giving prices at a conference? Anytime we need a random number this site is always used. How great would be to show a QRcode (OOB) on the screen that takes to a webpage capable of bootstrapping a DIDComm communication with everyone on the presentation, and then follow some protocol to randomize and decide on a winner? This would work for virtual conferences. The user would have the guarantee that the winner is really random, and not just another thing that I need to blindly trust if I want to participate.)
  • I don't understand why everyone is only looking into DIDComm to transfer verifiable credentials. yeah, that is a nice use case it can be sold to big entities and organizations. But this only wouldn't be enough to gain attraction IMO. Therefore would be obsolete before people start using it everywhere.

In summary, the need for a test suite is crucial to ensure interoperability between different systems or components by verifying their compliance with the same set of standards, protocols, and specifications.

[rodolfomiranda]

That's a great topic to also be discussed at the UG calls.

[FabioPinheiro]

Related with https://github.com/decentralized-identity/didcomm-messaging/issues/435