ePSIC-DLS / epsic_tools

Code for conversion and analysis of Merlin-Medipix data
GNU General Public License v3.0
6 stars 7 forks source link

Fix 1bit raw import #12

Closed M0hsend closed 5 years ago

M0hsend commented 5 years ago

Merlin / Medipix 1 bit RAW file is written with binary values packed into uint8 bytes. The header format is the same as regular uint8 file format. This code separates the header from the binary frames and unpacks the binary bits.

dnjohnstone commented 5 years ago

Is this something that needs to be updated in pyxem too? It would probably be preferable to maintain only one version of this code to avoid this kind of duplication issue...

M0hsend commented 5 years ago

Is this something that needs to be updated in pyxem too? It would probably be preferable to maintain only one version of this code to avoid this kind of duplication issue...

Yes, definitely. I was going to do a PR for this on pyxem shortly. I think maintaining this code in one place is a good idea - it would make my life a lot easier. But I think we would like to have flexibility on what this code does and change it accordingly - not having to wait on SciComp to update the pyxem install on the cluster whenever there is a change. What do you think? @chrisallen80

dnjohnstone commented 5 years ago

I agree that there is a balance with flexibility and stability. I personally think that the "conversion" or preprocessing script should always be separate and flexible and not require a potentially lengthy install. But I think the file reader should be stable enough that it can go in to a package and nearly never need modification.

The issue of course is that the file reader is not properly finished or tested. So what I'm really advocating for is sorting out the file reader properly and putting it somewhere stable before worrying about the pipeline. One might argue that a manufacturer selling this hardware should be able to provide a stable reader for their data...