filecoin-project / filecoin-chain-archiver

Filecoin snapshot / chain export software
Other
5 stars 3 forks source link

feat: support cloudflare r2 upload backend #49

Closed travisperson closed 2 years ago

travisperson commented 2 years ago

This removes the use of the website redirect option as it's not supported on Cloudflare's R2 service. The website redirect metadata value was only used to store the latest snapshot. However, we were already writing this as the value of the latest object. So we now just read this value in the index-resolver service.

Additionally, R2 does not respond with the location and instead simply returns auto. With S3 and minio we were taking avtange of this response value as the S3 API can also be a public-facing endpoint to download objects. This is not the case with R2 and the S3-compatible API is not the same endpoint that is used for public download.

To keep things fairly simple we are now providing this endpoint during upload. In the future though we may want to instead configure this on the index-resolver service. This would probably work better with an additional feature of R2 (custom-domains).