lquerel / gcp-bigquery-client

GCP BigQuery Client (Rust)
Apache License 2.0
92 stars 60 forks source link

JobApi query() method only returns partial results #28

Open adensur opened 1 year ago

adensur commented 1 year ago

It is really unclear from the comments/documentation that this method leaves off some data if query result is big enough

rahul-soshte commented 1 year ago

Check this document where they mention about this. only 10 seconds until the query runs and then it sends results. https://docs.rs/gcp-bigquery-client/latest/gcp_bigquery_client/model/query_request/struct.QueryRequest.html#structfield.timeout_ms

chroth7 commented 6 months ago

I do have the same problem, changing timeout_ms does not help (also, the number of returned rows is the same in all reruns... so it seems to be some other flag, rather than a timeout)

also, max_results has no effect (if increased)

chroth7 commented 6 months ago

fwiw, I did move to a JobConfigurationQuery approach, just like in the pagination example, and that works now (setting a sensible, small pagination size)