search

hotosm / galaxy-api

Backend to fetch data from Underpass
https://galaxy-api.hotosm.org/latest/redoc
GNU Affero General Public License v3.0
14 stars 5 forks source link

Stop the process and exit gracefully if shapefile export exceed more than 4 GB : Rawdata #245

Closed kshitijrajsharma closed 2 years ago

kshitijrajsharma commented 2 years ago

Plan is to instead of waiting for ogr2ogr for the big exports with shapefile as file format , we will monitor the process and stop it if it contains more than 4 GB of binding file size since limit for ogr2ogr is 4GB https://gdal.org/drivers/vector/shapefile.html

ramyaragupathy commented 2 years ago

Also pass the message to the end user that the export has failed for the specific format because of file size limitiations and encourage them to download this data in other formats. If there are other output formats selected as part of export and those formats succeeded, they should be available for download.

May be we could include an additional status PARTIALLY COMPLETED to capture such scenarios.

cc @itskshitiz321