Bug: Saving a blob with InputStream

[benjaminglatzeder]

CBL: 2.6 CE

Saving a blob with ByteArrayInputStream which extends from InputStream

new Blob("image/jpeg", getPicture(bitmap)
// ...
public static ByteArrayInputStream getPicture(Bitmap bitmap) {
ByteArrayInputStream myByteArrayInputStream = new ByteArrayInputStream(// ...);
// ...
return myByteArrayInputStream;
}

an error is thrown at SG: [INF] SyncMsg: c:[5b1db747] #39: Type:rev --> 400 Incorrect data sent for attachment with digest: sha1-lcJZ/lBqNbS23v0aIzI+LmSxwxA= Time:66.291689ms

and in Android Studio it's logged as:

E/CouchbaseLite/REPLICATOR: {N8litecore4repl6PusherE#1} Got error response to rev '7JFlD2OEV7U2ZBYiBXCC20aEaIL2::khj8xbs2yw3n::item::b55bea1b804a' #4-23fae3d30ea8398c9750e39637d52618ac7a3068 (seq #545): HTTP 400 'Incorrect data sent for attachment with digest: sha1-lcJZ/lBqNbS23v0aIzI+LmSxwxA='

If I wrap the ByteArrayInputStream with org.apache.commons.io.IOUtils.toByteArray there is no error and the blob is pushed up.

new Blob("image/jpeg", IOUtils.toByteArray(getPicture(bitmap))

[bmeike]

Yeah.... Looks to me like the code will read, at most 8K bytes. If your picture is bigger than that, you lose.

(Note that there is no such limit if you do the conversion to a byte array, yourself. The constructor that takes a byte array as its second arg will happily deal with an array of any size.)

    private byte[] getBytesFromInitialContentStream() {
        final ByteArrayOutputStream out = new ByteArrayOutputStream();

//...

            final byte[] buffer = new byte[MAX_CACHED_CONTENT_LENGTH];
            try {
                int bytesRead;
                while ((bytesRead = initialContentStream.read(buffer)) >= 0) {
                    out.write(buffer, 0, bytesRead);
                }
                contentResult = out.toByteArray();
                blobLength = contentResult.length;
                if (blobLength <= MAX_CACHED_CONTENT_LENGTH) {
                    content = contentResult;  // cache for later re-use
                }
            }
// ...
        return contentResult;
    }

[bmeike]

Tracked in https://issues.couchbase.com/browse/CBL-390

[bmeike]

Note that if that stream is longer than MAX_CACHED_CONTENT_LENGTH, the second call to this function will return different contents.

[bmeike]

Hey... .Can you reproduce this? Do you have code that will do it?

[benjaminglatzeder]

Here's the minimal project: CblDemo2.zip

There are two buttons. One to create the blob via InputStream and one which wraps the InputStream with org.apache.commons.io.IOUtils.toByteArray

The Sync Gateway log with inline comments is below. Please scroll to the right to see the comments.

2019-10-19T14:30:29.442+02:00 [INF] SyncMsg: c:[313c3ae3] #4: Type:proposeChanges #Changes: 1
2019-10-19T14:30:29.879+02:00 [INF] SyncMsg: c:[313c3ae3] #5: Type:rev   --> 400 Incorrect data sent for attachment with digest: sha1-dZ14HLe+G/ztZUmIAwsOt7wsiiU= Time:430.611679ms <-- clicked on button #1
2019-10-19T14:30:29.887+02:00 [INF] SyncMsg: c:[313c3ae3] #6: Type:setCheckpoint Client:cp-DNgLNESLiEdbBB653I2A2Kmfx3s=
2019-10-19T14:30:35.961+02:00 [INF] SyncMsg: c:[313c3ae3] #7: Type:proposeChanges #Changes: 1
2019-10-19T14:30:36.507+02:00 [INF] SyncMsg: c:[313c3ae3] #8: Type:rev   --> 400 Incorrect data sent for attachment with digest: sha1-dZ14HLe+G/ztZUmIAwsOt7wsiiU= Time:537.29246ms <-- clicked on button #1
2019-10-19T14:30:36.518+02:00 [INF] SyncMsg: c:[313c3ae3] #9: Type:setCheckpoint Client:cp-DNgLNESLiEdbBB653I2A2Kmfx3s= Rev:0-1
2019-10-19T14:30:38.834+02:00 [INF] SyncMsg: c:[313c3ae3] #10: Type:proposeChanges #Changes: 1
2019-10-19T14:30:39.394+02:00 [INF] SyncMsg: c:[313c3ae3] #11: Type:rev   --> 400 Incorrect data sent for attachment with digest: sha1-dZ14HLe+G/ztZUmIAwsOt7wsiiU= Time:550.823412ms <-- clicked on button #1
2019-10-19T14:30:39.403+02:00 [INF] SyncMsg: c:[313c3ae3] #12: Type:setCheckpoint Client:cp-DNgLNESLiEdbBB653I2A2Kmfx3s= Rev:0-2
2019-10-19T14:30:41.380+02:00 [INF] SyncMsg: c:[313c3ae3] #13: Type:proposeChanges #Changes: 1
2019-10-19T14:30:41.898+02:00 [INF] SyncMsg: c:[313c3ae3] #14: Type:rev   --> 400 Incorrect data sent for attachment with digest: sha1-dZ14HLe+G/ztZUmIAwsOt7wsiiU= Time:509.747548ms <-- clicked on button #1
2019-10-19T14:30:41.908+02:00 [INF] SyncMsg: c:[313c3ae3] #15: Type:setCheckpoint Client:cp-DNgLNESLiEdbBB653I2A2Kmfx3s= Rev:0-3
2019-10-19T14:30:45.090+02:00 [INF] SyncMsg: c:[313c3ae3] #16: Type:proposeChanges #Changes: 1
2019-10-19T14:30:45.714+02:00 [INF] CRUD: c:[313c3ae3]  Added attachment "sha1-dZ14HLe+G/ztZUmIAwsOt7wsiiU="
2019-10-19T14:30:45.728+02:00 [INF] SyncMsg: c:[313c3ae3] #18: Type:setCheckpoint Client:cp-DNgLNESLiEdbBB653I2A2Kmfx3s= Rev:0-4 <-- clicked on button #2 ... from now on both buttons work. Both upload
2019-10-19T14:30:45.740+02:00 [INF] Cache: c:[313c3ae3] GetCachedChanges("!", 1) --> 0 changes valid from #1
2019-10-19T14:30:45.740+02:00 [INF] Cache: c:[313c3ae3] GetCachedChanges("*", 1) --> 1 changes valid from #1
2019-10-19T14:30:45.749+02:00 [INF] Sync: c:[313c3ae3] Sent 1 changes to client, from seq 2
2019-10-19T14:30:45.794+02:00 [INF] SyncMsg: c:[313c3ae3] #19: Type:setCheckpoint Client:cp-DNgLNESLiEdbBB653I2A2Kmfx3s= Rev:0-5
2019-10-19T14:30:47.634+02:00 [INF] SyncMsg: c:[313c3ae3] #20: Type:proposeChanges #Changes: 1
2019-10-19T14:30:47.733+02:00 [INF] Cache: c:[313c3ae3] GetCachedChanges("!", 2) --> 0 changes valid from #1
2019-10-19T14:30:47.733+02:00 [INF] Cache: c:[313c3ae3] GetCachedChanges("*", 2) --> 1 changes valid from #3
2019-10-19T14:30:47.740+02:00 [INF] SyncMsg: c:[313c3ae3] #22: Type:setCheckpoint Client:cp-DNgLNESLiEdbBB653I2A2Kmfx3s= Rev:0-6
2019-10-19T14:30:47.745+02:00 [INF] Sync: c:[313c3ae3] Sent 1 changes to client, from seq 3
2019-10-19T14:30:47.790+02:00 [INF] SyncMsg: c:[313c3ae3] #23: Type:setCheckpoint Client:cp-DNgLNESLiEdbBB653I2A2Kmfx3s= Rev:0-7
2019-10-19T14:30:48.945+02:00 [INF] SyncMsg: c:[313c3ae3] #24: Type:proposeChanges #Changes: 1
2019-10-19T14:30:49.031+02:00 [INF] Cache: c:[313c3ae3] GetCachedChanges("!", 3) --> 0 changes valid from #1
2019-10-19T14:30:49.031+02:00 [INF] Cache: c:[313c3ae3] GetCachedChanges("*", 3) --> 1 changes valid from #4
2019-10-19T14:30:49.041+02:00 [INF] SyncMsg: c:[313c3ae3] #26: Type:setCheckpoint Client:cp-DNgLNESLiEdbBB653I2A2Kmfx3s= Rev:0-8
2019-10-19T14:30:49.046+02:00 [INF] Sync: c:[313c3ae3] Sent 1 changes to client, from seq 4
2019-10-19T14:30:49.090+02:00 [INF] SyncMsg: c:[313c3ae3] #27: Type:setCheckpoint Client:cp-DNgLNESLiEdbBB653I2A2Kmfx3s= Rev:0-9

The Sync Gateway function is absolutely minimal:

function sync_gateway_fn(doc, oldDoc) {
}

Guests are allowed in the Sync Gateway config file.

[bmeike]

Fixed in 2.7.0