Explore using S3 buckets to store data indexed by GeoServer

alexander-petkov / wfas

A placeholder for the WFAS project.

5 stars 1 forks source link

Explore using S3 buckets to store data indexed by GeoServer #48

Open wmjolly opened 2 years ago

wmjolly commented 2 years ago

The AWS implementations of GeoServer would benefit from being able to manage add data to GeoServer without having to constantly worry about filling up the disk. As this implementation evolves, so will the data needs. Let's put some time into learning how S3 can be used inside GeoServer: what are the strengths and weaknesses.

https://docs.geoserver.org/latest/en/user/community/s3-geotiff/index.html

alexander-petkov commented 2 years ago

Use the COG plugin

Use the COG plugin for reading images hosted on S3.

alexander-petkov commented 2 years ago

Add external overviews

Add external overviews to all COG files, in order for the COG plugin to work.

gdaladdo -ro 202109280000.tif 2 \
0...10...20...30...40...50...60...70...80...90...100 - done.

alexander-petkov commented 2 years ago

Performance comparison of S3 vs EBS-hosted mosaic over 100 iterations using CFS data: s3 vs EBS performance