search

rice-solar-physics / pydrad

Python tools for setting up HYDRAD runs and parsing output
https://pydrad.readthedocs.io
MIT License
4 stars 3 forks source link

Feature request: Convert HYDRAD output to single H5 file #115

Closed jwreep closed 3 years ago

jwreep commented 3 years ago

That is, take some permutation of .phy, .hstate, .ine, etc. files over some time range and convert it to a single H5 file. This would make sharing results significantly easier if we could deliver a single file.

This is also a lot easier to do in Python instead of writing a C-module.

What do you think?

wtbarnes commented 3 years ago

Yes, I think this is a really good idea! I think this is sort of what I had in mind in #96? But maybe it needn't be that complicated. Actually, I think I have a good idea for what this would look like. I'm going to open a PR with a prototype before I forget 😅

jwreep commented 3 years ago

Oops, missed that issue before making this post. Yes, same idea, though.

wtbarnes commented 3 years ago

Should these be interpolated onto a uniform grid with each variable stored as a time-distance array? Or should each timestep be it its own dataset?

sjbradshaw commented 3 years ago

All the file output is done in CAdaptiveMesh::WriteToFile() in mesh.cpp, so this function can be modified (or another one written and overloaded) to write the new format. I’ve been thinking about this for a while to reduce redundancy and also to label the data so that people know what the quantities, units, etc., are.

Cheers,

Steve

From: Will Barnes notifications@github.com Sent: Wednesday, September 30, 2020 3:27 PM To: rice-solar-physics/pydrad pydrad@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [rice-solar-physics/pydrad] Feature request: Convert HYDRAD output to single H5 file (#115)

Yes, I think this is a really good idea! I think this is sort of what I had in mind in #96 https://github.com/rice-solar-physics/pydrad/issues/96 ? But maybe it needn't be that complicated. Actually, I think I have a good idea for what this would look like. I'm going to open a PR with a prototype before I forget 😅

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/rice-solar-physics/pydrad/issues/115#issuecomment-701625900 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACC6C7XB5SVF24D4WNVH7GLSIOIAXANCNFSM4R7QPW2A . https://github.com/notifications/beacon/ACC6C7WYT2QVQSUG2A47HUTSIOIAXA5CNFSM4R7QPW2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOFHI7MLA.gif

jwreep commented 3 years ago

I would say that this should be to deliver the raw data, so do it without interpolation. The rest of the pydrad code could still interpolate afterwards, no?

jwreep commented 3 years ago

I also agree with Steve that appending the units on the data would make things much clearer, particularly for new users of HYDRAD.