search

jgliss / pyplis

Python toolbox for the analysis of UV SO2 camera data
GNU General Public License v3.0
7 stars 5 forks source link

add regexp filename parser, refactor filename parsing code for testability #18

Closed johannjacobsohn closed 5 years ago

johannjacobsohn commented 5 years ago

I found the current method of file name parsing somewhat cumbersome, so I've added a new method that uses a grouped regular expression. This should be more flexible and easier to work with. I've refactored CameraBaseInfo.py from utils.py to make tests easier and moved much of the filename-parsing code into testable pure functions.

I've binned identify_camera_from_filename which does not seem to be in use any more but created some headaches dependency wise.

If have more file name patterns that are in use I would gladly add them to the test-suite.

johannjacobsohn commented 5 years ago

I've developed this ontop of the py3 branch but rebased it on master so that its easier to merge, but I could move it to py3 again if desired.

solvejgdinger commented 5 years ago

I agree, it could be a good idea to include several kind of test images. If your image files aren't too large, you could upload them to the data folder. The images, I work with, have file sizes larger above 1GB and I wouldn't upload them therefore.

solvejgdinger commented 5 years ago

I can merge it into the py3 branch. Jonas wants to check the python 3 version first with his private scripts before releasing a new version on the master branch.

On the long term, I would like to separate the Camera class in two seperate classes. One which handles the read-in as flexible as possible ( filename, meta/header or pixel encoded information) and one which takes care of the physical parameters and additional camera-specific functions e.g. distortion correction. But I haven't started working on this yet.

johannjacobsohn commented 5 years ago

Could you provide just the filename pattern and maybe individual small test files (I you have those?). I thinks both methods can be moved from class to class w/o issues, they are not dependant on class state.

I think it is wise to test this as much as humanly possible with all current private scripts and shouldn't be rushed. Just tell me if you want this PR on top of py3 or master.

solvejgdinger commented 5 years ago

I have a special case of file name I guess. I have only one file per 200-400 images with filename alla 20190211Z1650_UV3A.fits all meta information is in the header and the exact time is encoded in the first 14 pixel of the image. So I am actually not using the information in the filename for anything. But I will try to create a test file with only 5 images.

jgliss commented 5 years ago

@johannjacobsohn I tested this PR some time ago, but tests were failing (pytest and / or example scripts) and I did not have time to further investigate. Needs further testing for now.

johannjacobsohn commented 5 years ago

I'll rebase on master, run the tests again and let you know

johannjacobsohn commented 5 years ago

I've rebased and fixed test failures, please review