Draft: Normalize AIA image, resize, and save as memory-mapped numpy array. - [opened]

[nabobalis]

In GitLab by @kingbob8 on Oct 30, 2020, 16:20

Merges aia2sdoml -> main

This script is to normalize the AIA image to 1 AU with specified disk size, correct for degradation and exposure time, and also provide an option to downsample the image and save it in memory mapped numpy array, which follows the same procedure as we did for the SDOML dataset.

[nabobalis]

In GitLab by @wtbarnes on Oct 31, 2020, 12:35

This is great! A few high level comments around how we actually want to integrate this functionality into aiapy.

w.r.t. the norm function,

• In keeping with the rest of our workflow in the package, I think the function should just do one thing: normalize the image to 1 AU.
• The combination of the steps (degradation, exposure time, 1 AU normalization) can then be put in an example in the gallery.
• The philosophy behind this is that aiapy provides individual chunks of functionality and users can then combine those chunks of work in whatever way they need. And we can of course give them examples of how best to do this!
• We should probably come up with a new name as norm is a bit vague. Since we already have normalize_exposure, maybe normalize_area? normalize_disksize?

Regarding the save_ml function,

• While I disagree with the choice of saving the data in that way, I understand that may be useful in some contexts (e.g. ML datasets where you may just want the images and loading FITS files with headers is overkill and often unnecessary). As above, I think this is better suited as an example in the gallery.
• A very naive question: how does the downsampling function in scikit-image differ from the resample function in sunpy? I think wherever possible, we should use existing methods in sunpy, but maybe it is inadequate here?

tl;dr my recommendations would be that,

1. norm be renamed and only do the 1 AU normalization procedure
2. Create a separate example showing how to combine degradation, exposure normalization, and the 1 AU correction
3. save_ml should be a separate example that shows how to save this data efficiently using numpy memmap and if possible, use the resample method from sunpy.

[nabobalis]

In GitLab by @kingbob8 on Nov 2, 2020, 10:26

Thanks, Will! That's great suggestions. I will separate the script into normalize_disksize function and the rest into an example gallery. For the downsampling function in scikit-image and in sunpy, I think the difference should be minimum. I will switch to resample for the example gallery.

[nabobalis]

changed target branch from master to main