search

zarr-developers / community

An open community with an interest in developing and using new technologies for tensor data storage.
18 stars 1 forks source link

Connecting with IRIS-HEP #30

Open alimanfoo opened 4 years ago

alimanfoo commented 4 years ago

Via @jpivarski:

Zarr is being mentioned in IRIS-HEP a lot more; it's being independently discovered by other physicists through other channels. Do you want to meet the larger group (virtually) at an IRIS-HEP topical meeting? This is a place where we discuss software that we can use and collaborate on for HEP.

Also relates to #29.

alimanfoo commented 4 years ago

Hi @jpivarski, I'd be very happy to join an IRIS-HEP meeting, would you like to propose a date? Should we wait until after Easter holiday season?

jpivarski commented 4 years ago

Sure! The usual meeting times are

and at the moment, the only slots taken are April 22 and April 27. Since Easter is a week from now, it would make sense to wait until after Easter Monday (April 13), as that's usually a CERN holiday. That leaves

as options.

alimanfoo commented 4 years ago

Thanks @jpivarski. I could do Monday April 20.

jpivarski commented 4 years ago

Monday, April 20 would be great!

This is a list of previous meetings, and it will be where the new meeting listed, once I make an entry for it. We usually post any materials (usually PDF of slides, but any link) here.

@lukasheinrich would also like to present at this meeting, since he has an Awkward ↔ Zarr converter (which I didn't know about, actually) and has been having good experience with it in his work; much better performance than HDF5.

alimanfoo commented 4 years ago

Cool, thanks @jpivarski! Should I assume folks will already be fairly knowledgeable about Zarr or would it help to present some introductory material (e.g., based on SciPy 2019 slides)? And/or present some info on work towards a new protocol spec (based on these slides)? How long should I budget for?

Great to hear of work towards zarr-awkward integration already.

alimanfoo commented 4 years ago

(Btw also happy not to present anything and just answer questions & discuss, whatever would work best for your community :smile:)

jpivarski commented 4 years ago

A few, like Lukas, already know about it, but I think the majority does not. Some introduction would be very helpful.

As an angle for the introduction, I think presenting Zarr as a long-term storage file format in competition with ROOT would be a hard sell (and might derail the discussion of Zarr itself), but there's been a lot of interest recently in using Zarr to deliver "fast turn-around" data for data subsets. (@lukasheinrich can correct me here, since he's actually doing these things.)

I'm personally interested in v3, and I think the others might be as well.

jpivarski commented 4 years ago

Here is an agenda:

https://indico.cern.ch/event/907792/

which may be adjusted between now and the 20th.

jpivarski commented 4 years ago

@alimanfoo and @lukasheinrich I hate to admit it, but I seriously goofed up by missing the upcoming IRIS-HEP meeting on Zarr. I just saw on my schedule that it starts in half an hour 20 minutes!

Because I didn't make the Friday announcement to the IRIS-HEP mailing lists that there would be a meeting on Monday, we won't get very high turn-out. Alistair would have to send his talk to me, but Lukas hasn't uploaded his talk either—have you two forgotten as well? If so, then I should reschedule and go on the Vidyo room to apologize to anybody who does show up.

Available slots are

Again, I'm really sorry about this! I hope it's not an inconvenience and I'll make sure it doesn't happen again.

alimanfoo commented 4 years ago

Hi @jpivarski, no problem at all, I had remembered but had (a) put it in my calendar for 1730 instead of 1630 after somehow messing up summer time and time zones, and (b) forgotten to send you slides in advance.

Any of the dates you give would be fine for me.

Slides I was planning to talk to are a combination of these (although I will cherry pick to stay under 30 mins):

jpivarski commented 4 years ago

Is April 29 okay? I'll try moving the event and adding your slides, and I'll try to get confirmation from @lukasheinrich to see if that works for him as well.

On the actual date, I'll contact you in advance to be sure that you can use Vidyo. (The web version works well for me, but it's always good to check.)

lukasheinrich commented 4 years ago

works for me as well.

jpivarski commented 4 years ago

I have rescheduled for April 29 and added @alimanfoo's talks. I'll send the announcement on the Monday before this Wednesday meeting, and we can test Vidyo shortly before that.

Thanks again and sorry for my mistake!

https://indico.cern.ch/event/907792/

alimanfoo commented 4 years ago

Thanks @jpivarski :+1:

lukasheinrich commented 4 years ago

Hi @alimanfoo -- sorry I somehow assumed this was just next Monday. If this is the only date for you that works, let's go with April 29th but would Monday May 11 also work for you? Apologies again!

jpivarski commented 4 years ago

@alimanfoo @lukasheinrich Hi, just checking in on this because I'm about to send the announcement for Wednesday, April 29. If you wouldn't mind moving the meeting to Monday, May 11, then we can include Lukas's talk on physicist experiences with Zarr, which would give more context. Otherwise, I'll send the announcement today that it will be on April 29.

Thanks!

alimanfoo commented 4 years ago

Hi @jpivarski, sorry I missed @lukasheinrich comment. If May 11 is better then that's fine for me.

jpivarski commented 4 years ago

Agh! I just checked the schedule (should have done it before writing the above) and all the Mondays are now taken until May 25. I don't think it makes sense to postpone for a month. Sorry, @lukasheinrich, I think we'll have to go ahead with the "originally revised" date of April 29.

Sorry for all the confusion.

lukasheinrich commented 4 years ago

That's fine. I'm happy to show my slides / usage some other time and we can ping @alimanfoo then

On Mon, Apr 27, 2020 at 2:27 PM Jim Pivarski notifications@github.com wrote:

Agh! I just checked the schedule (should have done it before writing the above) and all the Mondays are now taken until May 25. I don't think it makes sense to postpone for a month. Sorry, @lukasheinrich https://github.com/lukasheinrich, I think we'll have to go ahead with the "originally revised" date of April 29.

Sorry for all the confusion.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/zarr-developers/community/issues/30#issuecomment-619954627, or unsubscribe https://github.com/notifications/unsubscribe-auth/AARV6A4XAZURVXAVL3B6IMLROV23PANCNFSM4MCIQ5XA .

jpivarski commented 4 years ago

Okay, so then we're on for April 29 (this Wednesday). I'll remove Lukas's talk from the schedule and send out the announcement.

alimanfoo commented 4 years ago

OK, no problem, look forward to hearing @lukasheinrich talk another time.

jpivarski commented 4 years ago

@alimanfoo Thank you for the nice talk! We'll have the video ready for @lukasheinrich soon.

Where should I introduce myself/start looking into how Awkward might be implemented as a v3 extension? https://github.com/zarr-developers/zarr-specs/pull/16?

alimanfoo commented 4 years ago

Pleasure :-) I raised a starter issue here, would be great to have your thoughts: https://github.com/zarr-developers/zarr-specs/issues/62