search

CartoDB / raster-loader

https://raster-loader.readthedocs.io
Other
15 stars 4 forks source link

change default `chunk_size` parameter and catch bigquery `403 Exceeded rate limits` to better explain the error to the user #100

Closed francois-baptiste closed 1 year ago

francois-baptiste commented 1 year ago

Default chunk_size is set to 100 pixelblocks per upload batch right now. For large raster it triggers a Bigquery error 403 Exceeded rate limits cause you are not allowed to update a table to often.

See https://github.com/CartoDB/raster-loader/issues/48#issue-1494348251 to reproduce

A good think would be to size the chunck in function of the raster size for large raster. Another good thing would be to catch this Bigquery upload error and to reraise an error to the user explaining that he should increase the chunk size

Jesus89 commented 1 year ago

Closed by https://github.com/CartoDB/raster-loader/pull/116