Zarr in the News - Githubissues

zarr-developers / community

An open community with an interest in developing and using new technologies for tensor data storage.

18 stars 1 forks source link

Zarr in the News #4

Open joshmoore opened 5 years ago

joshmoore commented 5 years ago

Similar to the ongoing conversation in "Regular zarr/n5 community conference calls" (https://github.com/zarr-developers/zarr/issues/315), this is an ongoing place to post blogs posts and another announcements about Zarr that everyone is aware of.

joshmoore commented 5 years ago

https://www.unidata.ucar.edu/blogs/news/entry/netcdf-and-native-cloud-storage

alimanfoo commented 5 years ago

Zarr Python 2.3 release on the Zarr blog.

alimanfoo commented 5 years ago

Zarr protocol v3 design update on the Zarr blog.

jakirkham commented 5 years ago

Loading image data with Dask and storing it in Zarr.

ref: https://blog.dask.org/2019/06/20/load-image-data

joshmoore commented 5 years ago

alimanfoo commented 5 years ago

Article about how pangeo is used at CNES:

Mention of Zarr:

Outside of Satellite imagery, Ifremer has used Pangeo on our cluster to analyze the outputs of a big ocean simulation (MitGCM LLC4320). One typical thing done on this simulation is the computation of time-frequency kinetic energy spectra, which basically build a big datacube over the globe and reorganize the data so as to have the full temporal axis on a given location in memory in order to perform the scientific computation on it. Xarray helps to do this in very few lines of code, and by using Zarr file format they were able to speed-up the data loading by a factor of ten compared to NetCDF.

alimanfoo commented 5 years ago

Zarr at SciPy 2019: talk, slides.

alimanfoo commented 5 years ago

An older post but still relevant, HDF in the cloud by Matt Rocklin.

alimanfoo commented 5 years ago

@jhamman's article about the CESM Large Ensemble which is now in the AWS Public Dataset Program in Zarr format.

alimanfoo commented 4 years ago

Zarr gets CZI EOSS funding!!!

alimanfoo commented 4 years ago

Zarr/CZI news piece on my institution's website.

joshmoore commented 4 years ago

Cross link between Zarr & Bioimaging EOSS funding

joshmoore commented 4 years ago

Zarr/N5 used to expedite transforming bio-imaging files into pyramidal OME-TIFF

jhamman commented 4 years ago

New climate model data now in Google Public Datasets

ryan-williams commented 4 years ago

@jakirkham presented this short deck at the CZI EOSS Kickoff Mtng today

ryan-williams commented 4 years ago

Loading NumPy arrays from disk: mmap() vs. Zarr/HDF5 is on Hacker News today (h/t @hammer)

jhamman commented 4 years ago

Streaming with Zarr - written by yours truly. Binder ready demo here:

jakirkham commented 4 years ago

That's great Joe! Thanks for sharing! How can we promote this so more people are aware of this use case? 🙂

jhamman commented 4 years ago

How can we promote this so more people are aware of this use case?

I think a zarr-examples repo would be an interesting way to highlight some of these more specialized use-cases. I'm thinking something like dask-examples.

joshmoore commented 1 year ago

Been quite for a while: I'll kick us off with https://xarray.dev/blog/xarray-kvikio :smile: