search

AndrewAnnex / planetcantile

tile matrix sets for other planets
BSD 3-Clause "New" or "Revised" License
9 stars 1 forks source link

COGs Best Practices #6

Open jlaura opened 1 year ago

jlaura commented 1 year ago

We discussed best practices for COG generation. I am opening this issue here because we can/should define the TMS that should be used for the COG at time of creation. Hopefully we can discuss here or move it to an appropriate place.

@vincentsarago wrote the following medium post. He is referencing the rio-cogeo project that he is the primary dev on.

Can we build out the planetary COG best practices using the best practices described above?

AndrewAnnex commented 1 year ago

@jlaura yeah I think we can discuss it here for the moment. I'll have to read @vincentsarago 's medium post and look at rio-cogeo, I used https://kokoalberti.com/articles/geotiff-compression-optimization-guide/ as a starting point to come up with some encodings I use. I tend to use zstd_zlev1_pred2 if I am compressing byte data or zstd_pred3 for DEMs, and then lossy LERC compression if I can afford it (example some hirise dems, ctx dems, derived products from dems)

AndrewAnnex commented 1 year ago

@jlaura read through the blog post a bit and the rio-cogeo code. looks like lerc compression is one of the profiles vincent has implemented which is good to know. I never used webp encoding as most of the data I work with is single band which does benefit from JPEG compression, while webp doesn't work well with grayscale data. Interestingly gdal tends to default to 256x256 tiles (including the COG driver) but I can see the tradeoff between get requests and tile size so I think it is semi use-case dependent.

AndrewAnnex commented 1 year ago

in general I think there is a table of data type and use-case one can make with the desired rio-cogeo profile, although we may want to add a profile or two for grayscale JPEG and few LERC compression levels (precisions of 0.01m, 0.1m, etc).

AndrewAnnex commented 1 year ago

@jlaura I've created a fork of rio-cogeo with a new branch and added some profiles. It may make sense to make another github project like planetcantile that just includes the new profiles and nothing else but for now this should be good enough to test out https://github.com/AndrewAnnex/rio-cogeo/tree/add_planetary_profiles

AndrewAnnex commented 1 year ago

now in PR form https://github.com/cogeotiff/rio-cogeo/pull/251

jlaura commented 1 year ago

@AndrewAnnex I added another profile for HiRISE. I am working on a big HiRISE ARD release (all RDRs) and using the proposed approach with LERC Z ERROR set to 0.0001.

AndrewAnnex commented 1 year ago

@jlaura looks like my rasterio patch was just merged in to add back LERC support (in 1.3.7 rasterio), so hopefully the rio-cogeo PR can be merged soon

AndrewAnnex commented 1 year ago

@jlaura as per the maintainers comments I've closed the PR for rio cogeo. We could move those profiles to planetcantile but I agree it isn't really necessary to update rio-cogeo itself. Otherwise I think this issue should move on to other topics around cog best practices, or maybe creating a blog post describing the various LERC profiles and what uses they are good for etc