Trailing zeros are trimmed after the data is hashed which brakes integrity checks

[yurii-uhlanov-intellecteu]

Description

JSON input with trailing zero(s) is being added to a Private Data Collection (PDC):

1. First, a JSON input with trailing zero(s) is hashed to record the hash on the ledger.
2. Next, the trailing zeros get trimmed resulting in a modified JSON.
3. This new value is stored in a PDC.
4. When retrieved from a PDC, its hash will no longer match the hash of the original input. This brakes integrity checks.

Example:

JSON Input before hashing

{"assetId":"BUG","value":1.400}

JSON stored in PDC

{"assetId":"BUG","value":1.4}

Tested on Fabric 2.3.0 + CouchDB 3.1.1, Fabric 2.5.0 + CouchDB 3.2.2

Steps to reproduce

Prerequisites

To demonstrate the issue, Fabric samples were used.

• sample network setup files located in fabric-samples/test-network
• sample java chaincode project located in fabric-samples/asset-transfer-private-data/chaincode-javawas modified

In the chaincode project:

• Genson dependency was added to build.gradle file: implementation group: 'com.owlike', name: 'genson', version: '1.6'

• All existing DataTypes and Contracts were replaced with the following Contract file:

  AssetContract.java

  ```java /* * SPDX-License-Identifier: Apache-2.0 */ package org.hyperledger.fabric.samples.privatedata; import com.owlike.genson.Genson; import org.hyperledger.fabric.contract.Context; import org.hyperledger.fabric.contract.ContractInterface; import org.hyperledger.fabric.contract.annotation.Contract; import org.hyperledger.fabric.contract.annotation.Default; import org.hyperledger.fabric.contract.annotation.Transaction; import org.hyperledger.fabric.contract.annotation.DataType; import org.hyperledger.fabric.contract.annotation.Property; import org.hyperledger.fabric.shim.ChaincodeException; import java.math.BigDecimal; import java.security.MessageDigest; import java.security.NoSuchAlgorithmException; import java.util.Objects; @Contract(name = "AssetContract") @Default public final class AssetContract implements ContractInterface { private static final String ASSET_COLLECTION_NAME = "assetCollection"; private static final String ASSET_ID_BUG = "BUG"; private static final String ASSET_ID_OK = "OK"; private static final String DIGEST_ALGORITHM = "SHA-256"; private final Genson genson = new Genson(); @Transaction(name = "WriteBug", intent = Transaction.TYPE.SUBMIT) public void writeBug(final Context ctx) { Asset asset = new Asset(ASSET_ID_BUG, new BigDecimal("1.400")); String assetJson = genson.serialize(asset); System.out.printf("Put asset: %s, collection: %s, ID: %s\n", assetJson, ASSET_COLLECTION_NAME, ASSET_ID_BUG); ctx.getStub().putPrivateData(ASSET_COLLECTION_NAME, ASSET_ID_BUG, assetJson); } @Transaction(name = "ValidateBug", intent = Transaction.TYPE.EVALUATE) public void validateBug(final Context ctx) throws Exception { byte[] assetJson = ctx.getStub().getPrivateData(ASSET_COLLECTION_NAME, ASSET_ID_BUG); byte[] realHash = MessageDigest.getInstance(DIGEST_ALGORITHM).digest(assetJson); byte[] expectedHash = ctx.getStub().getPrivateDataHash(ASSET_COLLECTION_NAME, ASSET_ID_BUG); System.out.println("Real json: " + new String(assetJson)); System.out.println("Real hash: " + new String(realHash)); System.out.println("Expected hash: " + new String(expectedHash)); if (!Objects.deepEquals(realHash, expectedHash)) { throw new ChaincodeException("Ledger hash is not equal to hash of private data"); } } @Transaction(name = "WriteOk", intent = Transaction.TYPE.SUBMIT) public void writeOk(final Context ctx) { Asset asset = new Asset(ASSET_ID_OK, new BigDecimal("1.4")); String assetJson = genson.serialize(asset); System.out.printf("Put asset: %s, collection: %s, ID: %s\n", assetJson, ASSET_COLLECTION_NAME, ASSET_ID_OK); ctx.getStub().putPrivateData(ASSET_COLLECTION_NAME, ASSET_ID_OK, assetJson); } @Transaction(name = "ValidateOk", intent = Transaction.TYPE.EVALUATE) public void validateOk(final Context ctx) throws ChaincodeException, NoSuchAlgorithmException { byte[] assetJson = ctx.getStub().getPrivateData(ASSET_COLLECTION_NAME, ASSET_ID_OK); byte[] realHash = MessageDigest.getInstance(DIGEST_ALGORITHM).digest(assetJson); byte[] expectedHash = ctx.getStub().getPrivateDataHash(ASSET_COLLECTION_NAME, ASSET_ID_OK); System.out.println("Real json: " + new String(assetJson)); System.out.println("Real hash: " + new String(realHash)); System.out.println("Expected hash: " + new String(expectedHash)); if (!Objects.deepEquals(realHash, expectedHash)) { throw new ChaincodeException("Ledger hash is not equal to hash of private data"); } } } @DataType() final class Asset { @Property() private final String assetId; @Property() private final BigDecimal value; Asset(final String assetId, final BigDecimal value) { this.assetId = assetId; this.value = value; } public String getAssetId() { return assetId; } public BigDecimal getValue() { return value; } } ```

To start the network and deploy the chaincode, execute:

export PATH=${PWD}/../bin:$PATH
export FABRIC_CFG_PATH=$PWD/../config/
export CORE_PEER_TLS_ENABLED=true
export CORE_PEER_LOCALMSPID="Org1MSP"
export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=${PWD}/organizations/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
export CORE_PEER_ADDRESS=localhost:7051

./network.sh up createChannel -ca -s couchdb
./network.sh deployCC  -ccp ../asset-transfer-private-data/chaincode-java -ccl java -ccn private -cccg ../asset-transfer-private-data/chaincode-java/collections_config.json

To reproduce the bug, execute:

peer chaincode invoke -c '{"function":"WriteBug","Args":[]}' -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile "${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem" -C mychannel -n private --peerAddresses localhost:7051 --tlsRootCertFiles "${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt" --peerAddresses localhost:9051 --tlsRootCertFiles "${PWD}/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt"
peer chaincode invoke -c '{"function":"ValidateBug","Args":[]}' -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile "${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem" -C mychannel -n private --peerAddresses localhost:7051 --tlsRootCertFiles "${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt" --peerAddresses localhost:9051 --tlsRootCertFiles "${PWD}/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt"

You should see this:

Error: endorsement failure during invoke. response: status:500 message:"Ledger hash is not equal to hash of private data" 

[denyeart]

When writing and re-reading JSON in any platform, the before and after are not guaranteed to match exactly, per the JSON spec. This is true with most JSON parsers, with CouchDB by itself, and when using CouchDB with Fabric.

The behavior is documented here: https://hyperledger-fabric.readthedocs.io/en/latest/couchdb_as_state_database.html#reading-and-writing-json-data

The intended use of GetPrivateDataHash() is to verify that a passed pre-image's hash matches a previously stored pre-image's hash on an organization's peer that doesn't have the private data, not to check against the re-retrieved value on a peer that has the private data, this is described here: https://hyperledger-fabric.readthedocs.io/en/latest/private-data/private-data.html?highlight=getprivatedatahash#sharing-private-data

Therefore this is working as expected and as documented. If you don't think the documentation describes it well enough, this issue can be used to clarify the documentation.

Note that you can accomplish what you want by not saving JSON to CouchDB. Simple string key/value pairs will remain the same when saved and re-read (using either LevelDB or CouchDB).