Errors with chunked encoding responses and @adobe/aio-lib-state or @azure/core-rest-pipeline

alexkli commented 2 years ago

When using @adobe/node-openwhisk-newrelic with @adobe/aio-lib-state (which internally uses @azure/core-rest-pipeline for CosmosDB requests), it will FAIL requests that come back with chunked encoding (and no content-length header).

SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at httpRequest (/code/node_modules/@azure/cosmos/dist/index.js:7773:85)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async execute (/code/node_modules/@azure/cosmos/dist/index.js:7614:26)
    at async ClientContext.getDatabaseAccount (/code/node_modules/@azure/cosmos/dist/index.js:8493:37)
    at async CosmosClient.getDatabaseAccount (/code/node_modules/@azure/cosmos/dist/index.js:8897:26)
    at async GlobalEndpointManager.resolveServiceEndpoint (/code/node_modules/@azure/cosmos/dist/index.js:8716:51)
    at async ClientContext.read (/code/node_modules/@azure/cosmos/dist/index.js:8153:34)
    at async Container.read (/code/node_modules/@azure/cosmos/dist/index.js:6269:26)
    at async Container.readPartitionKeyDefinition (/code/node_modules/@azure/cosmos/dist/index.js:6324:41)'

The http instrumentation, namely this code https://github.com/adobe/node-openwhisk-newrelic/blob/cbbd5f86e9780cea3dea559653056618722d84f4/lib/probe/http-client.js#L250-L253

seems to be incompatible with the way the Azure http client library handles requests.

Sample code to reproduce:

const stateLib = require('@adobe/aio-lib-state')
const {NewRelic} = require('@adobe/openwhisk-newrelic')

async function main(params){

    const metrics = new NewRelic({
        newRelicEventsURL: params.newRelicEventsURL,
        newRelicApiKey: params.newRelicApiKey
    });

    try {
        let state = await stateLib.init();
        await state.put('12345', "dummy - value");
        const res = await state.get('12345');
        return {
            statusCode: 200,
            body: {data: res}
        };
    } catch (e){
        return {
            statusCode:500,
            body: {data: e}
        }
    } finally {
        await metrics.activationFinished();
    }
}

exports.main = NewRelic.instrument(main)

Error is triggered in aio-lib-state here: https://github.com/adobe/aio-lib-state/blob/master/lib/impl/CosmosStateStore.js#L34

alexkli commented 2 years ago

Initial fix was #57 but that change completely removed tracking of response size for chunked responses, so we reverted it in #58.

A working solution is in #61.

adobe-bot commented 2 years ago

:tada: This issue has been resolved in version 1.0.7 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

adobe / node-openwhisk-newrelic

Errors with chunked encoding responses and @adobe/aio-lib-state or @azure/core-rest-pipeline #59