Off-chain storage

[srderson]

A requirement of Openchain is the concept of off-chain storage. Off-chain storage will allow chaincode developers to store large files outside of the blockchain while referencing the files from a transaction stored in the blockchain.

For example, there may be a case where a chaincode is a codified written contract or represents a portion of a written contract. In this case we would like to 'link' the chaincode to the contract. Another scenario is the case where a transaction may be associated with a signed document. The signed document may be very large and we do not wish to store it across peers, but instead would like to record a verifiable link to the document in the transaction.

The verifiable link to a file from a transaction contains 2 parts.

• A URL to the file.
• A hash of the file.

While the URL does not guarantee future retrieval of the file, the hash does allow one to verify whether a file is the original file associated with the transaction.

As mentioned by @jeffgarratt in an earlier discussion, the goal of off-chain storage should not be to build a file storage repository. There are many existing storage repositories that would work for our use case. The most obvious example may be Git, although there are many other systems.

The goal of off-chain storage is to

1. Allow a user to associate documents or arbitrary and potentially large files with a transaction.
2. Store the URL and the hash of the files with the transaction on the blockchain.

   Proposal

There exists an API on peers to invoke a transaction. This transaction either deploys a new chaincode or invokes a function in a chaincode. This API should expose a parameter which is an array of one of more URLs which point to files that are to be associated with the contract. Here are the steps for associating a file with a transaction.

1. A client sends a transaction to a peer, specifying 0 or many URLs associated with the transaction.
2. The transaction is sent to the block packaging peer.
3. The block packaging peer performs a GET on each URL. It expects a 200 response code with a response body. If the response code is not a 200 or the response body is empty, the transaction is marked as invalid.
4. Assuming a valid transaction, the block packaging peer hashes the response body and stores this hash in the transaction.
5. The transaction is sent to all validating peers.
6. Each validating peer also performs a GET on the URL, checking for a 200 response code and response body. Assuming a valid response, they again hash the response body and compare it with the hash added to the transaction by the block packaging peer. Assuming a match, they will vote 'yes' during the consensus. A non-match would result in a 'no' vote.
7. Assuming a successful consensus, the validating peers add the block containing the transaction which in turn contains the URL and hash to their blockchain.
   • The above proposal requires the existence of an external central storage repository.
   • The API for sending a transaction could also accept a file directly. The peer providing the API would have to be configured with the external storage URL. The peer would then upload the file to the off-chain storage and add the URL to the transaction before sending the transaction to the block packaging peer.

[binhn]

I think the feature should provide APIs, like

• Store Document: Store and return the hash of the document
• Store Documents: Store an array of documents and return the array of hashes
• Get Document: Return the document identified by the hash
• Get Documents: Return the documents identified by the array of hashes

The implementation of the APIs could interface with DocuSign, Cloundant, or other services. The applications would be responsible for storing the document(s) and associating them with the transaction(s). The validators should not verify the integrity of the off-chain documents, but it would be the responsibility of the application or an external entity.

[srderson]

What benefit would the Openchain APIs provide over just using the Cloudant, DocuSign, Git, etc. API directly? I'm not sure if all these services give you the file hash (Git does) but that's really just an extra line or two of code in the application.

[binhn]

We would provide a consistent API for applications to use whether with Cloudant or DocuSign or other services. It also ensures that the document returned is the document represented by the hash.

If the implementation is for DocuSign, it doesn't need to hash the documents since DocuSign does that, but Cloudant or others that might not guarantee integrity then the API implementation for those would provide hashing.

The API could also be enhanced to provide signing and verifying signatures; ie, more encapsulation of what DocuSign provides.

[frankyclu]

What we need is to have a 100% reliable side storage for clients to store large contracts files. Some clients may want to use docusign/cloudant while others maybe more comfortable with their local database.

Our goal should be creating a protocol (or client sdks) that allow users to easily choose their preferred way of storing contract documents, whether that’s docusig/cloudant service or local sql/nosql database.

[binhn]

move to backlog

[jwolpert]

At the blockchain meetup in SF on Monday, OneName and Blockstore showed their approach to mutable and immutable data stores, the latter using DHTs. Just fyi.

[christo4ferris]

Isn't the etherium approach also DHT based?