search

weecology / EvergladesTools

Computational tools for working with Everglades image data
MIT License
0 stars 2 forks source link

rio -v mbtiles too slow but fast with resampling #170

Open henrykironde opened 8 months ago

henrykironde commented 8 months ago

Fast

rio -v mbtiles /blue/ewhite/everglades/projected_mosaics/webmercator/2020/Vacation/Vacation_05_07_2020_projected.tif \
-o /blue/ewhite/everglades/mapbox/2020/Vacation/Vacation_05_07_2020.mbtiles \
--zoom-levels 17..24 -f PNG \
--tile-size 512 \
--tile-driver 'jpeg' \
--resampling 'average'

Extremely slow:

rio -v mbtiles /blue/ewhite/everglades/projected_mosaics/webmercator/2020/Vacation/Vacation_05_07_2020_projected.tif -o /blue/ewhite/everglades/mapbox/2020/Vacation/Vacation_05_07_2020.mbtiles --zoom-levels 17..24 -f PNG
ethanwhite commented 8 months ago

Interesting! How does the fast one look at max zoom?

ethanwhite commented 8 months ago

What version of rio-mbtiles is this with? I'm trying it for some tree work this morning and on the current release I'm seeing --format instead of --tile-driver (--tile-driver errored)

henrykironde commented 8 months ago

I am at 1.3.9

ethanwhite commented 8 months ago

I think that's the rasterio version not the rio-mbtiles version

henrykironde commented 8 months ago

rio-mbtiles==1.6.0

ethanwhite commented 8 months ago

OK, that's the same version as mine, but that makes me more confused about why you seem to have --tile-driver as an option and I don't. I don't see --tile-driver here https://github.com/mapbox/rio-mbtiles

henrykironde commented 8 months ago

You are right that was introduced by mistake

rio -v mbtiles /blue/ewhite/everglades/projected_mosaics/webmercator/2020/Vacation/Vacation_05_07_2020_projected.tif \
-o /blue/ewhite/everglades/mapbox/2020/Vacation/Vacation_05_07_2020.mbtiles \
--zoom-levels 17..24 -f PNG \
--tile-size 512 \
--resampling 'average'

The main takeaway was the use of resampling taking much less time(We don't want to resample in out case). However it turns out that they do recommend that rio-v mbtiles works better with smaller files. Trying to see if we have other options but so far all are out dated. like https://github.com/ecometrica/gdal2mbtiles

The main takeaway is that resampling significantly reduces processing time. However it's not applicable to our specific case. It turns out that they recommend that rio-v mbtiles performs better with smaller files. We are currently exploring alternative options, but many of them seem outdated, such as the one available at https://github.com/ecometrica/gdal2mbtiles.

ethanwhite commented 8 months ago

OK, thanks! I'm running a 40 GB geotiff for trees now so we'll see how that goes.