search

ercius / openNCEM

A collection of packages and tools for electron microscopy data analysis supported by the National Center for Electron Microscopy facility of the Molecular Foundry
GNU General Public License v3.0
60 stars 28 forks source link

Unit tests #6

Closed fniekiel closed 4 years ago

fniekiel commented 7 years ago

We need to overhaul the unit tests, to allow convenient collaboration of multiple coders.

Let's put up a list of requirements and current status, which we can use to discuss and work on these things.

  1. Storage of binary data:

    • needs to be outside github
    • currently on peters google drive?

  2. Structure of binary data:

    • currently badly organized in ncempy/test/resources

  3. Test data:

    • should be small and handy
    • for file formats should cover ideally all possible combinations (data types, software versions, etc.)
    • Peter started with some nice SER/EMI files

  4. Implementation:

    • all codes must be tested
    • how to test properly?
fniekiel commented 7 years ago

So my thoughts are:

  1. I am fine with google drive. We should work on keeping the files small, though. This means I should not use full 2k diffraction images to test SER reading capabilities ...

  2. I am unhappy with having the files kind of hidden in ncempy/test/resources. Maybe we should have a top level folder with test/example files in the openNCEM folder even, so they can also be used by matlab etc.

  3. Lets collect as many combinations as possible. I guess this collection will grow with time, as more people start to use the code. Whenever something does not work, we probably get to add some more files.

  4. My unit tests right now mainly test for proper input data to the algorithms rather than testing whether they run correctly yielding the right results. I add test cases, which run test data against known outcome. For the file formats I am still wondering about the best way. We probably should compare the read in data to some reference, which could be an emd version of the same data. But do we only compare the data or also the meta data? For me the testing usually was converting to an EMD file and than have a look at the data, meta data and the structure.

ercius commented 7 years ago
  1. Yes. We should use much smaller data files to test things. Im planning to generate a number of test DM3 files to test the DM class. Ill upload them to google drive when that is done.
  2. Agreed. The same files will be needed for matlab
  3. I can generate these since I have access to the original programs and the microscopes that generate them. Ill use either real data (in the case it can be used in the tools too) or just tiny test files (32x32 pixel image of all 1's for example).
  4. Can you add a test case to open an image and test it? We could test for simple meta data (i.e. pixel size) to keep it simpler. Ive never used unit tests so I need some hand holding to get started.
ercius commented 7 years ago

I added a DM3 file and a DM4 file to the Google Drive folder which both contain the same data and are 3D in size. The data was reported as 63,64,65 in size. Im not sure how that translates to contiguous data on disk. I also added calibration data so we can be sure to match the calibration data to the proper dimension. See the accompanying image file.

ercius commented 4 years ago

This is a very old issue. Im updating the tests to use pytest and will distribute small test data in the ncempy/data folder.