search

2i2c-org / docs

Documentation for 2i2c community JupyterHubs.
https://docs.2i2c.org
9 stars 17 forks source link

Document use of cloud object storage #226

Closed jnywong closed 4 months ago

jnywong commented 7 months ago

Follows on from #215

Related material

### Tasks
- [x] Review NASA Openscapes Python example https://github.com/NASA-Openscapes/earthdata-cloud-cookbook/pull/325
- [ ] https://github.com/2i2c-org/infrastructure/issues/4214
- [ ] https://github.com/2i2c-org/infrastructure/issues/4216
- [x] Reference [NASA Openscapes examples](https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/using-s3-storage.html) in the Service Guide once it's merged
yuvipanda commented 7 months ago

Just as a cautionary note for

Add how-to guides for extracting temporary credentials to upload/download to a bucket from a local machine

Doing this will always incur cloud egress fees for the project, which can be very large and very quick - https://infrastructure.2i2c.org/topic/billing/chargeable-resources/#egress-fees. For LEAP they hit about 1k$ in less than a day of folks doing that. This is a helpful thing to document, but needs to come with cost warnings around it.

jnywong commented 7 months ago

🙀

yuvipanda commented 7 months ago

And just to clarify, this only accounts for download - upload is free. Download pricing depends on where they're downloading it to. The cloud billing topic doc linked has more information.

jnywong commented 7 months ago

Haha got it yes, ingress vs egress 💰 I'm going to include all of the warning callouts