spacetelescope / jwst

Python library for science observations from the James Webb Space Telescope
https://jwst-pipeline.readthedocs.io/en/latest/
Other
545 stars 158 forks source link

Median image derivitive calculation during outlier detection assumes 0 values outside image. #8402

Open braingram opened 3 months ago

braingram commented 3 months ago

The abs_deriv function included in outlier detection: https://github.com/spacetelescope/jwst/blob/415de860dddb67e94fc01ba1441579dcce1434ee/jwst/outlier_detection/outlier_detection.py#L472 Compute absolute differences between adjacent pixels in the median image: https://github.com/spacetelescope/jwst/blob/415de860dddb67e94fc01ba1441579dcce1434ee/jwst/outlier_detection/outlier_detection.py#L426 which is used in computation of the cr_mask: https://github.com/spacetelescope/jwst/blob/415de860dddb67e94fc01ba1441579dcce1434ee/jwst/outlier_detection/outlier_detection.py#L437 and https://github.com/spacetelescope/jwst/blob/415de860dddb67e94fc01ba1441579dcce1434ee/jwst/outlier_detection/outlier_detection.py#L446

When computing the differences in abs_deriv edge effects are handled by assuming that edge pixels abut 0 values (so the difference between an edge pixel and a non-existent neighboring pixel off the edge is equal to the value of the edge pixel).

For the following toy array:

>> arr = np.arange(9).reshape((3, 3))
array([[0, 1, 2],
       [3, 4, 5],
       [6, 7, 8]])

The function returns:

>>> jwst.outlier_detection.outlier_detection.abs_deriv(arr)
array([[3., 3., 3.],
       [3., 3., 5.],
       [6., 7., 8.]])

Note that for pixel [2, 2] with a value of 8, the maximum absolute difference between adjacent pixels is 2 for all non-edge differences but the result is 8 due to the edge treatment.

I suspect that this leads to increased outlier pixel flagging for edge pixels. However if this is intended this issue can be closed.

braingram commented 3 months ago

@jemorrison @tapastro Is this edge treatment as-intended?

braingram commented 3 months ago

Also I believe the following would ignore edge differences (so don't include a difference between an edge pixel and an off-edge pixel 0 value in the difference):

def abs_deriv(array):
    deriv = np.zeros(array.shape, dtype=np.float64)
    delta_0 = np.abs(np.diff(array, axis=0))
    delta_1 = np.abs(np.diff(array, axis=1))
    deriv[1:] = delta_0
    deriv[:-1] = np.maximum(deriv[:-1], delta_0)
    deriv[:, 1:] = np.maximum(deriv[:, 1:], delta_1)
    deriv[:, :-1] = np.maximum(deriv[:, :-1], delta_1)
    return deriv

Using the above function and the toy array the result is an array of all 3s.