Closed vuphamcs closed 4 months ago
I just discovered that resultSetMetaData
is only provided by Snowflake's API when retrieving data for the first partition (partition 0). That would explain why Avalanche.Steps.GetPartitions.build_status_request/4
stores avalanche_row_types
, so it can reuse rowTypes
to decode later partitions. I think this means there is no bug here but is there any advice on how to best go about retrieving very large query results in smaller increments?
Greetings and thank you for providing this library. I have just started using Avalanche and have encountered an error when attempting to retrieve query results using the
partition
option. Please let me know if this is a bug or if I am using the library incorrectly.Describe the bug After submitting an async sql statement and retrieving the results with the partition option,
Avalanche.Steps.DecodeData.decode_data/1
errors because row_types is nil.To Reproduce Steps to reproduce the behavior:
Error message
Expected behavior The result should have properly decoded rows from partition 1.
Additional context I am using the partition option because the query result is too large to store in memory. When I run the status query without specifying
[partition: 1]
, all results are returned and decoded properly. When[partition: 1]
is passed the response body seems to be missingresultSetMetaData
whererowType
information is usually found.