vrodgom / statmorph

Python code for calculating non-parametric morphological diagnostics of galaxy images.
BSD 3-Clause "New" or "Revised" License
42 stars 15 forks source link

smoothness and _sky_smoothness are 0 when boxcar_size = 1 #2

Closed kdpenner closed 6 years ago

kdpenner commented 6 years ago

_sky_smoothness is 0 when boxcar_size = 1, which is the case for all of my sources. This is because ndi.uniform_filter(bkg, size=boxcar_size) returns bkg, which causes the difference between the two to be 0.

vrodgom commented 6 years ago

I'm not sure this is an issue with the code. According to eq. (11) from Lotz, Primack & Madau (2004; see also Conselice 2003), the image should be smoothed by a boxcar of width 0.25*rpetro. If the Petrosian radius is smaller than 8 pixels, then boxcar_size = 1, so I would say that the source is too compact to have a reliable measurement.

kdpenner commented 6 years ago

The problem may be with cutout_extent. If it has the default value of 1.5, the skyboxes for my images are small, usually 2 or 4 pixels wide. If I use cutout_extent = 10, the skyboxes are bigger, and I get some non-zero smoothness values. But if cutout_extent is too large, the Petrosian radii for some of the detections is large. (Because the detections are spurious?)

vrodgom commented 6 years ago

This sounds like an issue with the input segmap, which might be too small to enclose the entire source. Since creating good segmaps is not trivial, I left this task up to the user. You can have a look at the tutorial for an example of how to create segmaps with photutils. You can also try more traditional software such as SExtractor.

kdpenner commented 6 years ago

The input segmap is an image much larger than the extent of the source. I've been using photutils to create segmaps (and export them). Do you have another idea for why the skyboxes would be too small in the sliced image?

vrodgom commented 6 years ago

It could also be the opposite, i.e., that the main segment of the segmap is so large that there isn't much background left (the skybox is supposed to be a region of the image with "pure" background).

You can have a look at the upper-left panel of this image. It shows the original image, along with the "postage stamp", whose size is determined by "cutout_extent" (red), and the skybox (blue).

EDIT: Or maybe the original image size is too small compared to the source.

kdpenner commented 6 years ago

I'm unsure if either of these explanations is the culprit. I've uploaded a notebook duplicating your plots with my data: https://github.com/kdpenner/morphology/blob/master/sky_smoothness.ipynb The first set of plots shows the input image and resulting segmap. The second set shows the result for one source in the input image using cutout_extent = 1.5. The third set shows the result for the same source in the input image using cutout_extent = 10.

vrodgom commented 6 years ago

Thanks for sharing the notebook. There is definitely something wrong with your input segmap. The purpose of the segmap is to label the regions of the image that belong to different sources. While I see a single source in your image, your segmap consists of ~8 small segments scattered across the image, some of which have just a few pixels. The "souce_morphology" function will calculate the morphologies of all labeled sources, even the ones with just a few pixels, and the skybox will be searched for in a region around each source (the "postage stamp") with a size that is comparable to the size of the source itself (determined by "cutout_extent"). This is why the skybox is so small, but this is just a symptom of the real problem, which is having a weird segmap.

kdpenner commented 6 years ago

Could you clarify the following statements:

"a size that is comparable to the size of the source itself (determined by "cutout_extent")"---this doesn't make sense to me. Let's pretend the blip is a real source. The size of the source is the size of the source, regardless of how large the cutout is.

"This is why the skybox is so small, but this is just a symptom of the real problem, which is having a weird segmap."---but when I use cutout_extent = 10 the skybox is a reasonable size.

vrodgom commented 6 years ago

"a size that is comparable to the size of the source itself (determined by "cutout_extent")" Here, I was referring to the size of the cutout (the "postage stamp"), which by default is 1.5 times the size of the source.

"but when I use cutout_extent = 10 the skybox is a reasonable size." Yes, because then the cutout is 10 times the size of the source, so there is a better chance of finding a reasonably sized skybox within the cutout.

kdpenner commented 6 years ago

There's still a disconnect somewhere. This is my understanding of the process:

1) the source size is estimated from the segmap (is how this is done important?) 2) a postage stamp of size cutout_extent * (size of source) is extracted from the input image 3) a skybox is placed on a region of pure background in the postage stamp

For the case I've uploaded, assuming the blip is real, how can the problem of zero smoothness when cutout_extent = 1.5 be due to the segmap? When I change cutout_extent, the only other variable in this exercise, I get nonzero smoothness because the skybox is a reasonable size.

vrodgom commented 6 years ago

The 3 steps you outlined are correct. In the first step, the "size" of the source corresponds to the minimal bounding box of the segment of interest, which is then multiplied by "cutout_extent" (by default 1.5) in step 2 to define the "postage stamp". However, there is at least another parameter involved in the smoothness calculation, which is "petro_fraction_cas" (by default 0.25).

The reason you get a zero smoothness with the default parameter values is simply that some of the sources are too small. From your segmap, it seems that some of the sources are just 3 pixels across, which (by default) would result in cutouts with just floor(1.53.0) = 4 pixels across. The Petrosian radius cannot be larger than the cutout and, as I explained at the beginning of this thread, if the Petrosian radius is smaller than 8 pixels, then boxcar_size = floor(petro_fraction_cas rpetro) = floor(0.25 * rpetro) = 0 or 1, which means that there is no smoothing.

If you increase the value of "cutout_extent", you might be able to measure a Petrosian radius that is at least 8 pixels and therefore get a non-zero smoothness measurement, but I would be very skeptical of a smoothness measurement (or any other morphological measurement) of a source that is just 4 pixels across. Hopefully the code returns flag=1 (i.e., bad measurement) for these sources?

kdpenner commented 6 years ago

Gotcha, the additional step of estimating the Petrosian radius is what I needed to know. Thanks.

As far as the (truly) fake blip is concerned, when cutout_extent = 1.5 its flag = 0 and S/N per pixel = 2.6. When cutout_extent = 10 its flag = 0 and S/N per pixel = 0.9.