search

astropy / astroscrappy

Speedy Cosmic Ray Annihilation Package in Python
70 stars 34 forks source link

cosmic rays at image edges #12

Open crawfordsm opened 8 years ago

crawfordsm commented 8 years ago

Hi @cmccully -- have you run into any problems with cosmic rays at edges? I'm using the default parameters, and it seems to be having some problems detecting cosmic rays at the edge of the image. Is there something else I should be setting or is there something obvious I am missing?

I can send some example code if that helps and or explore it a little further.

cmccully commented 8 years ago

Hi @crawfordsm. Sorry for the delay. I have been at AAS. I think I originally (intentionally) ignored cosmic rays within a few pixels of the edges. I found that it was producing more artifacts than finding true cosmic rays. I didn't think this would be an issue as the edges of images are often rife with issues. I am open to suggestions though. How close to the edges were you considering?

crawfordsm commented 8 years ago

No worries! It was actually just in a test script that I had written and the random ones that were at the edge of the pixels were not being detected. These were typically within a few pixels of the edges. If it's a known issue that you have looked into then I wouldn't worry about it, but it might be worthwhile to add a note to the documentation.

jehturner commented 7 years ago

I have also run into this. It's the only thing that doesn't seem to work as well as the original lacos_spec (even after setting sepmed=False and cleantype=medmask). Since I have a mosaic of data from 12 CCD amplifiers on 3 chips, it's not just the edges of the final spectra that are affected.

jehturner commented 7 years ago

There is some evidence that the CR residuals I still see after changing sepmed & cleantype are due to differences between the background fitting I do before running astroscrappy and the IRAF version. I believe I wrote that before FittingWithOutlierRemoval was available in modeling, so will try updating my code to use that and see whether it resolves the problem.

jehturner commented 6 years ago

The extra residuals I was seeing close to edges (compared with lacos_spec in IRAF) were partly due to my Python background subtraction, which I have now fixed after some changes to astropy.modeling. However, astroscrappy is still leaving some extra residuals within a few (<5) rows/columns of edges, even with sepmed=False, fsmode="median" and cleantype='medmask.

This happens both at the array edges and also next to columns that are flagged as bad in inmask and interpolated over in indat. It only affects a small proportion of pixels (eg. ~100 out of 2M) but is not ideal when cleaning 12 amplifiers that then get stitched together with extra residuals on both sides of each boundary. I haven't looked into this further to date, but thought I'd report back on what I'm seeing...

jehturner commented 6 years ago

OK, I confirm that the extra CR residuals I'm seeing at the edges of each amplifier seem to be due to the reason @cmccully mentioned -- that astroscrappy's median filters leave 1-3 rows or columns unfiltered at each edge of the image (containing the original data values instead of a median).

I have a patch that makes the 5x5 filter behave exactly the same as the IRAF implementation, which causes edge CRs to be identified properly (they don't get cleaned fully yet, but masking them is the main problem). This still needs extending to the 3x3 & 7x7 filters, but I'll post what I have in case you have immediate comments. It may be that the parallelization could be better optimized (I'm not really conversant with that stuff), but I believe I've got the OMP-related bugs out.