search

pangeo-data / storage-benchmarks

testing performance of different storage layers
Apache License 2.0
12 stars 1 forks source link

Brief Update: March 24, 2018 #17

Closed kaipak closed 6 years ago

kaipak commented 6 years ago

Just a little update here. @jreadey and I have been slowly refining the prototype for the benchmarks and have a few synthetic tests: Zarr to local, Zarr to GCS, h5netcdf/HDS to local. Also, we've chosen test data for the benchmarks (LOCA data as indicated in issue #1 ).

We're currently in discussion on infrastructure on where the benchmarks will be run and how we can store persistent results from the heterogeneous environments as well as fleshing out the rest of the tests. I'm considering deploying a Docker container in GCS environment for this, and @jreadey is looking into doing something similar in AWS.

kaipak commented 6 years ago

Also, I've updated the readme in this repo to reflect the test cases we've identified that were previously strewn across issues comments. Please feel free to add more if I missed anything.