search

ptycho / ptypy

Ptypy - main repository
Other
36 stars 16 forks source link

PtydScan needs a review (info vs meta) #115

Open bjoernenders opened 6 years ago

bjoernenders commented 6 years ago

Current fixes led to weird patches where crucial information is stored in info/num_frames_actual. or info/positions_scan although the info is not meant to be part of the .ptyd spec according to http://ptycho.github.io/ptypy/rst/data_management.html#ptyd-file-format

This needs fixing. However my approach would be to use meta/num_frames ... But I do remember this was the original way. And apparently that caused some trouble leading to this info workaround. Could somebody remind me what the problem was ?

alexbjorling commented 6 years ago

Sorry I am pressing the wrong buttons.

What I wanted to say was that we've been working on both PtydScan and meta in release_candidate.

bjoernenders commented 6 years ago

I took a look at it and part of our problems was that we were not adhering to the spec in http://ptycho.github.io/ptypy/rst/data_management.html#ptyd-file-format

Also I reverted many of the changes that came with 94a6257.

pierrethibault commented 6 years ago

Would it be a big deal to just store the whole of info as meta? In a way, there is no such thing as too much metadata. I feel that it might be a mistake to make a rigid prescription for meta that we need to update every time something comes up.

bjoernenders commented 6 years ago

The compatibility with .ptyd format was my main concern. Also meta is created at a different point in the processing pipeline and is there to accurately reflect what is stored in the .ptyd. The info was supposed to reflect the input parameters to PtyScan and is the general (more) mutable container. I know it seems tedious but I rather separate the input to what is generated to the spec of .ptyd.

bjoernenders commented 6 years ago

There will be people that would want to build their on .ptyd from scratch. They would need to know what needs to get in there. And if they compare it with one of .ptyds from examples they ask themselves, why there is so much other stuff in meta if it's not essential.

bjoernenders commented 6 years ago

Admittedly I made one patch that did exactly that.... assuming a certain key in info even though info was not essential according to the spec. And If I read the spec, than I wouldn't have done it, because I already anticipated that this very thing belongs. So that wasn't really clever of me,, I only had the code in front of me and not the doc.

bjoernenders commented 6 years ago

This issue is postponed. Also it acts under the hood, so it shouldn't change the API much