Chunk going missing

[mbwilding]

Hey, sometimes the first chunk in a large query will not be included in the aggregated final result. Have you experienced this? Potentially a SF issue but unsure. Running in a AWS .NET 6 lambda.

[fixer-m]

@mbwilding Hi! Well, that's weird. I tried different queries and always got the first chunk data. I would suggest to check total rows count in response metadata and rows count that aggregated result contains (they should be equal):

var result = await _snowflakeClient.QueryRawResponseAsync("select top 10000 * from SNOWFLAKE_SAMPLE_DATA.TPCH_SF1000.SUPPLIER;");

var chunksDownloadInfo = new ChunksDownloadInfo() { ChunkHeaders = result.ChunkHeaders, Chunks = result.Chunks, Qrmk = result.Qrmk };
var parsed = await ChunksDownloader.DownloadAndParseChunksAsync(chunksDownloadInfo);

var totalRowCountInChunks = result.Chunks.Sum(c => c.RowCount);
Assert.AreEqual(totalRowCountInChunks, parsed.Count); 

As well as you can check rows in a first chunk from raw response - does it actually contains first rows from query result? Or it contains rows from result with some offset? (That will prove that SF just didn't returned first chunk and returned second instead, for instance).

[mbwilding]

Here is what I mean. Happening in windows too apparently. 0.4.0 added to project so I can debug, usually use the nuget.

[mbwilding]

[fixer-m]

@mbwilding Hi! This issue was fixed in 0.4.1. Turns out that SF API sometimes returns rows in "chunks" and in a "rowset" at the same time.

[fixer-m]

Fixed in 0.4.1.