Kusto query failed in Teams web service with error "Invalid character in header content ["x-ms-app"]"

dylanw-oss commented 8 months ago

Describe the bug Kuso query failed in Teams web service with error message ""Invalid character in header content ["x-ms-app"]". I'm not sure if this is a Teams Framework or Kusto bug. The same code works in local testing but failed when running in Teams web service in Azure. https://github.com/microsoft/teams-ai/issues/1443

To Reproduce Add a Kusto query function to Teams bot, and I explicitly set "x-ms-app" header with code as below, but not working. Can anyone help me understand if this is the right way to setup "x-ms-app" header? or in which situation kustoClient.execute throw this exception "Invalid character in header content ["x-ms-app"]"? how to fix it? Appreciated!

const executeKustoQuery = async (scope: string, query: string): Promise<KustoResultTable[]> => {
    const crp = new ClientRequestProperties({
        application: "teamsbot",
        clientRequestId: `TeamsBot.${scope};${uuidv4()}`,
    });

    const results = await kustoClient.execute(database, query, crp);
    return results.primaryResults;
};

Expected behavior no exception

Screenshots Got this error in Azure log stream:

[onTurnError] unhandled error: TypeError [ERR_INVALID_CHAR]: Invalid character in header content ["x-ms-app"]
TypeError [ERR_INVALID_CHAR]: Invalid character in header content ["x-ms-app"]
at ClientRequest.setHeader (node:_http_outgoing:579:3)at new ClientRequest (node:_http_client:262:14)
at Object.request (node:https:353:10)
at dispatchHttpRequest (C:\home\site\wwwroot\node_modules\axios\dist\node\axios.cjs:2932:21)
at C:\home\site\wwwroot\node_modules\axios\dist\node\axios.cjs:2622:5at new Promise (<anonymous>)
at wrapAsync (C:\home\site\wwwroot\node_modules\axios\dist\node\axios.cjs:2602:10)
at http (C:\home\site\wwwroot\node_modules\axios\dist\node\axios.cjs:2640:10)
at Axios.dispatchRequest (C:\home\site\wwwroot\node_modules\axios\dist\node\axios.cjs:3623:10)
at Axios._request (C:\home\site\wwwroot\node_modules\axios\dist\node\axios.cjs:4002:33)
at Axios.request (C:\home\site\wwwroot\node_modules\axios\dist\node\axios.cjs:3877:41)
at processTicksAndRejections (node:internal/process/task_queues:96:5)

at async KustoClient._doRequest (C:\home\site\wwwroot\node_modules\azure-kusto-data\dist-esm\src\client.js:186:29)
at async executeKustoQuery (C:\home\site\wwwroot\lib\controllers\CountController.js:22:21)
at async getCountData (C:\home\site\wwwroot\lib\controllers\CountController.js:77:12)
at async C:\home\site\wwwroot\lib\index.js:103:21at async Object.handler (C:\home\site\wwwroot\node_modules\@microsoft\teams-ai\lib\AI.js:135:20)
at async AI.run (C:\home\site\wwwroot\node_modules\@microsoft\teams-ai\lib\AI.js:329:34)
at async C:\home\site\wwwroot\node_modules\@microsoft\teams-ai\lib\Application.js:450:21
at async Application.run (C:\home\site\wwwroot\node_modules\@microsoft\teams-ai\lib\Application.js:353:16) {code: 'ERR_INVALID_CHAR'}

Setup:

SDK Version: "^5.2.3"
- node version: v18
- typescript version: "^5.3.3"

Browser Setup:

framework: Teams web
server env: Teams web
browser name/version: Version 123.0.2420.53 (Official build) stable app, beta channel (64-bit)Version

Desktop:

OS: Windows 11
Version: 26085.1

Additional context ...

Luciasea commented 7 months ago

I ran into the same issue when using azure-kusto-data V5. It blocks me to get a daily report generated by kusto data and hosted on function app. Wondering if there is a workaround and the ETA for the fix.

AsafMah commented 7 months ago

We have just released v6.0.1 which should have a fix for this, does it still happen to you in this version?

Luciasea commented 7 months ago

Hi AsaMah,

I updated the package to V6.0.1, unfortunately, I still get the same error when testing it in my function app. [A screenshot of a computer Description automatically generated]

I can confirm I'm using the latest package in my code and uploaded it successfully to my function app. The following is the package.json file in azure-kusto-data in node_moudles folder that I downloaded from my function app, I it shows the package version V6 as expected. [A screen shot of a computer code Description automatically generated]

From: AsafMah @.> Sent: Monday, April 8, 2024 9:49 PM To: Azure/azure-kusto-node @.> Cc: Lufeng Ren @.>; Comment @.> Subject: Re: [Azure/azure-kusto-node] Kusto query failed in Teams web service with error "Invalid character in header content ["x-ms-app"]" (Issue #299)

We have just released v6.0.1 which should have a fix for this, does it still happen to you in this version?

- Reply to this email directly, view it on GitHubhttps://github.com/Azure/azure-kusto-node/issues/299#issuecomment-2044142357, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AXCQPKWC2ISKZHZPV53LYBDY4NXNFAVCNFSM6AAAAABFIIK3IWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBUGE2DEMZVG4. You are receiving this because you commented.Message ID: @.***>

JoostLambregts commented 7 months ago

edit: tl;dr: A workaround for the issue is reverting to an older node version.

I am running into a similar issue with the x-ms-app header, but I have some useful additional information.

We are running in an Azure function app
The code used to work fine, but stopped working today after Azure upgraded to node 18.19.1
After switching the node version back to v16, the code started working again

I will try tomorrow with the 6.0.1 version and see if that helps.

edit: here is the relevant part of our code:

    kcs = KustoConnectionStringBuilder.withSystemManagedIdentity(clusterConnectionString)
    const kustoClient = new KustoClient(kcs)
    const clientRequestProps = new ClientRequestProperties()
    const oneMinute = 1000 * 60
    clientRequestProps.setTimeout(oneMinute)
    clientRequestProps.clientRequestId = `${config.applicationName};${getGuid()}`
    clientRequestProps.setOption('Application', config.applicationName)
    const results = await kustoClient.execute(database, query, clientRequestProps)

edit2: The issue is still present in version 6.0.1

ssc-di commented 7 months ago

We have the same problem since three days, the problem seems to be that the default value for x-ms-app uses process.title (https://github.com/Azure/azure-kusto-node/blob/master/packages/azure-kusto-data/src/clientDetails.ts#L36), which contains the whole node command line with invalid chars.

On our azure function app the header looks like this: 'x-ms-app': '"node" "C:\\Program Files (x86)\\SiteExtensions\\Functions\\4.31.0\\workers\\node\\dist/src/nodejsWorker.js" --host 127.0.0.1 --port 50705 --workerId b362f703-4028-48e4-a945-fae900b8262a --requestId c1508d5b-9737-4ed1-b955-fa5fefc0d388 --grpcMaxMessageLength 2147483647 --functions-uri http://127.0.0.1:50705/ --functions-worker-id b362f703-4028-48e4-a945-fae900b8262a --functions-request-id c1508d5b-9737-4ed1-b955-fa5fefc0d388 --functions-grpc-max-message-length 2147483647ݡ韐ਵ\x03' (Checked with KustoConnectionStringBuilder.withSystemManagedIdentity(ADX_URL).clientDetails().getHeaders()

AsafMah commented 7 months ago

I'm re-opening, we have just released 6.0.2 that should fix this issue.

Please let us know if you're still having it after upgrading.

JoostLambregts commented 7 months ago

@AsafMah just upgraded, errors no longer occur. Thanks for the speedy resolution.

dylanw-oss commented 7 months ago

@AsafMah, I verified it with package 6.0.2 and this error no longer occur. 🎉🎉🎉 Thanks for the speedy resolution!

Azure / azure-kusto-node

Kusto query failed in Teams web service with error "Invalid character in header content ["x-ms-app"]" #299