Store trace files in a configurable S3 bucket

acrlabs / simkube

Record-and-replay Kubernetes simulator based on KWOK

https://appliedcomputing.io/simkube

MIT License

164 stars 7 forks source link

Store trace files in a configurable S3 bucket #61

Closed drmorr0 closed 1 week ago

drmorr0 commented 7 months ago

Description

We need to be able to store trace files in someplace that is not just "local disk". The obvious first step here is being able to have sk-tracer export to an S3 bucket somewhere, and then have sk-driver pull down from that bucket. As a part of this ticket, please include an example IAM configuration in the docs that people can use.

I'm not 100% sure, but this may require changes to either the tracer API or the Simulation CRD; if so please make sure you read Making API changes.

drmorr0 commented 2 weeks ago

TIL about https://github.com/apache/arrow-rs/tree/master/object_store (https://docs.rs/object_store/latest/object_store/), we should investigate if we can use that.