Closed M0hsend closed 5 years ago
Is this something that needs to be updated in pyxem too? It would probably be preferable to maintain only one version of this code to avoid this kind of duplication issue...
Is this something that needs to be updated in pyxem too? It would probably be preferable to maintain only one version of this code to avoid this kind of duplication issue...
Yes, definitely. I was going to do a PR for this on pyxem shortly. I think maintaining this code in one place is a good idea - it would make my life a lot easier. But I think we would like to have flexibility on what this code does and change it accordingly - not having to wait on SciComp to update the pyxem install on the cluster whenever there is a change. What do you think? @chrisallen80
I agree that there is a balance with flexibility and stability. I personally think that the "conversion" or preprocessing script should always be separate and flexible and not require a potentially lengthy install. But I think the file reader should be stable enough that it can go in to a package and nearly never need modification.
The issue of course is that the file reader is not properly finished or tested. So what I'm really advocating for is sorting out the file reader properly and putting it somewhere stable before worrying about the pipeline. One might argue that a manufacturer selling this hardware should be able to provide a stable reader for their data...
Merlin / Medipix 1 bit RAW file is written with binary values packed into
uint8
bytes. The header format is the same as regularuint8
file format. This code separates the header from the binary frames and unpacks the binary bits.