DM3 class

[ercius]

I just uploaded a working DM3 class to io on the development branch. I have tested this with a single image and an image series file. Some more extensive testing is needed. To do:

1) Further data types (RGB, complex, spectra, etc.) will be added soon in the getDataset function to bring DM3.py to full capability as with my dm3Reader.m matlab code. 2) Cleaning up the code to be comparable to ser.py 3) Add a DM4 class which is fairly easy conversion from the DM3 class. I might put these in the same DM.py to share code. We can then use validDM3 and validDM4 to read dm3 and dm4 with the same dm.py file in io.

Let me know if you find any issues or have any questions.

[fniekiel]

Hi, good job! Works for Karens DM3 diffraction patterns from Titan.

I ran across a few things, which are completely up to discussion:

• I have called the SER and EMD classes fileSER and fileEMD. I am not completely happy with that naming, but we should do it consistently. I wanted to avoid doubling the name of the .py file, as it could lead to confusion. Yours currently is ncempy.io.dm3.DM3(...). Do you have a better naming idea?

• Are you planning to incorporate a writeEMD() into DM3? That was my solution for SER. I favor to do the EMD conversion in the particular fileformat class rather than in the EMD class, as it gives more freedom on the specific implementation. The alternative way would be to put it into the EMD class and create a function which takes an abstract datafile object. We would than have to create this abstract class which SER and DM3 inherit from, to enforce implementation of things like number_of_elements and getDataset(). The latest when it comes to filling the EMD file with all the meta data available, this probably breaks.

• We should put up unit testing for DM3 but there is some more discussion involved as we should overhaul the current tests. I will open a separate issue.

[ercius]

I have called the SER and EMD classes fileSER and fileEMD. I am not completely happy with that naming, but we should do it consistently. I wanted to avoid doubling the name of the .py file, as it could lead to confusion. Yours currently is ncempy.io.dm3.DM3(...). Do you have a better naming idea?

I meant to change the DM3 class to something different than DM3. I agree that we should use a different class name. fileDM3 and fileDM4 will work for me. I will change that the next chance I get.

Are you planning to incorporate a writeEMD() into DM3? That was my solution for SER. I favor to do the EMD conversion in the particular fileformat class rather than in the EMD class, as it gives more freedom on the specific implementation. The alternative way would be to put it into the EMD class and create a function which takes an abstract datafile object. We would than have to create this abstract class which SER and DM3 inherit from, to enforce implementation of things like number_of_elements and getDataset(). The latest when it comes to filling the EMD file with all the meta data available, this probably breaks.

I think it is useful to have the converter to EMD in each file class. However, some common functions (such as creating dim vectors, etc.) could be kept in the EMD class. What do you think of that? The abstract data object would be the best, but maybe that is more of a long term goal. It could take a lot of work to abstract all the types of data we might want to write. We should get some experience with working with all of these different types of data before we try to handle it all in an EMD converter.

[fniekiel]

I agree with all your points. I am having difficulties thinking of particular common functions to go into EMD, but I definitely agree that they belong their. I have one dim helper function in SER to translate TIAs "offset, delta, element" thingy into a vector. I thought that is specific to SER but in fact I have never worked with anything else?

[ercius]

Ok. Ill try to think of common things that might live in EMD. You might be correct that its mostly custom code in each case.

The TIA offset, delta, element is not unique to TIA. I think DM will have a similar set of meta data. Most people ignore the offset and element, except for spectra where the offset is commonly needed. We should try to use all three when available.

[ercius]

I added the dm3dm4 branch to work on this outside of dev.

dm3 class name is changed to fileDM3 to match other io

dm4.py now works. I tested it with a single 2D image. Next I need to test 3D and 4D data.

[ercius]

I added 3D capabilities to dm3.py. I t works as expected.

I added 3D and 4D capabilities to dm4.py. 3D works. I do not yet have a 4D data set to test.

I want to combin dm3 and dm4 classes into one dm.py file. It will make it easier to maintain and add to. Ill leave this open until that is done.

[ercius]

I combined the dm3.py and dm4.py into dm.py. Most functions are working as expected but this needs to be tested more on different files. This currently works for the dm3 and dm4 files on Google Drive and some other experimental data sets I have tested.

This branch is probably good to merge into development

[ercius]

dm.py is ready for the development branch. We should test it out on a few things and make sure its feature complete. But, Ill close this now since its basically working as expected.