Progressive mode in azure-kusto-data

[wmgries]

Hi folks,

I have a two part question: in looking at the C# and Rust examples, and the Query v2 REST API doc, it looks like Kusto supports a streaming model called "progressive" mode. First, is this the correct understanding of these APIs?

Second, does the azure-kusto-data Node SDK support progressive mode? Playing around my own code and a large data set, the SDK seems to wait until all the results are downloaded, rather than streaming the results. Passing results_progressive_enabled as a request option in a similar way to the C# example seems to result in errors.

Here's some sample code, more or less lifted from the example, to demonstrate:

import { 
    Client as KustoClient, 
    KustoConnectionStringBuilder,
    ClientRequestProperties as KustoClientRequestProperties,
} from "azure-kusto-data";

const clusterName = "<mycluster>";
const databaseName = "<mydatabase>";
const queryStr = "MyTable | where value == 10";

const connStr = KustoConnectionStringBuilder.withAzLoginIdentity(
    `https://${clusterName}.kusto.windows.net`);
const client = new KustoClient(connStr);

const requestProperties = new KustoClientRequestProperties();
requestProperties.setOption("results_progressive_enabled", true);

// error happens here
const response = await client.executeQuery(databaseName, queryStr, requestProperties);

The produced output is very long, so I just copied the initial error code.

{
  code: "Z_DATA_ERROR",
  errno: -3,
  message: "incorrect header check",

....

This code does work when I omit the results_progressive_enabled request property.

Thanks,

Will Gries

[yogilad]

It's part of our plans, but there's no current ETA for it.

[mcgear]

Any news on this?

[AsafMah]

We are starting to work on this, mainly @alonadam implementing, still no ETA.