Detect Large Documents in Cosmos Sink

[sklose]

Cosmos does not support documents larger than 2MB. Currently the Output Sink can go into an infinite loop when a larger document is being written. We should be able to detect that case and bail the retry in that case.

[sklose]

this is related to #58

[chrnola]

If it's any help, here's an example event from our logs showing the CosmosDB API returning an HTTP 400 when attempting to create a document that is too large:

{
    "time": "2020-02-06T20:15:14.899Z",
    "level": "EROR",
    "ctx": "XXXX",
    "fields": {
        "msg": [
            "CosmosOutputSink: Create Document returned an error on attempt 98/Infinity, next retry in 30000 ms"
        ],
        "err": [
            {
                "code": 400,
                "body": "{\"code\":\"BadRequest\",\"message\":\"Message: {\\\"Errors\\\":[\\\"Encountered exception while executing function. Exception = Error: DB Query returned FALSE: createDocument failed on document at index: 0, stream_id: 7637fda95323843750ffb183a0146106, sn: 81.\\\\r\\\\nStack trace: Error: DB Query returned FALSE: createDocument failed on document at index: 0, stream_id: 7637fda95323843750ffb183a0146106, sn: 81.\\\\n   at tryCreate (script.js:34:13)\\\\n   at Anonymous function (script.js:19:13)\\\\n   at Anonymous function (script.js:616:29)\\\"]}\\r\\nActivityId: c3f27ba2-ee8d-4748-bec1-3833d12925bb, Request URI: /apps/a46235ed-1a53-4c89-a8e9-d794eafe62dc/services/36d8f3be-4bed-4329-a202-30f37980db43/partitions/09989c31-b9b8-492e-bdbe-f9014aff60f0/replicas/132252792054514944p/, RequestStats: \\r\\nRequestStartTime: 2020-02-06T20:15:14.6745692Z, RequestEndTime: 2020-02-06T20:15:14.8545731Z,  Number of regions attempted:1\\r\\nResponseTime: 2020-02-06T20:15:14.8545731Z, StoreResult: StorePhysicalAddress: rntbd://cdb-ms-prod-eastus2-fd21.documents.azure.com:14086/apps/a46235ed-1a53-4c89-a8e9-d794eafe62dc/services/36d8f3be-4bed-4329-a202-30f37980db43/partitions/09989c31-b9b8-492e-bdbe-f9014aff60f0/replicas/132252792054514944p/, LSN: 4851228, GlobalCommittedLsn: 4851228, PartitionKeyRangeId: 103, IsValid: True, StatusCode: 400, SubStatusCode: 0, RequestCharge: 309.18, ItemLSN: -1, SessionToken: -1#4851228, UsingLocalLSN: False, TransportException: null, ResourceType: StoredProcedure, OperationType: ExecuteJavaScript\\r\\n, SDK: Microsoft.Azure.Documents.Common/2.9.2\"}",
                "headers": {
                    "transfer-encoding": "chunked",
                    "content-type": "application/json",
                    "server": "Microsoft-HTTPAPI/2.0",
                    "x-ms-last-state-change-utc": "Tue, 04 Feb 2020 08:40:15.327 GMT",
                    "lsn": "4851228",
                    "x-ms-schemaversion": "1.9",
                    "x-ms-quorum-acked-lsn": "4851228",
                    "x-ms-current-write-quorum": "3",
                    "x-ms-current-replica-set-size": "4",
                    "x-ms-xp-role": "1",
                    "x-ms-global-committed-lsn": "4851228",
                    "x-ms-number-of-read-regions": "0",
                    "x-ms-transport-request-id": "6002",
                    "x-ms-cosmos-llsn": "4851228",
                    "x-ms-cosmos-quorum-acked-llsn": "4851228",
                    "x-ms-session-token": "103:-1#4851228",
                    "x-ms-request-charge": "309.18",
                    "x-ms-serviceversion": "version=2.9.0.0",
                    "x-ms-activity-id": "c3f27ba2-ee8d-4748-bec1-3833d12925bb",
                    "strict-transport-security": "max-age=31536000",
                    "x-ms-gatewayversion": "version=2.9.2",
                    "date": "Thu, 06 Feb 2020 20:15:14 GMT",
                    "x-ms-throttle-retry-count": 0,
                    "x-ms-throttle-retry-wait-time-ms": 0
                },
                "activityId": "c3f27ba2-ee8d-4748-bec1-3833d12925bb"
            }
        ]
    }
}