Unexpected limitation of measurement accuracy for lengthscales larger than pixel dimensions

[oskooi]

As an experiment to investigate the accuracy of imageruler, we can measure the separation distance between two circles which contain no sharp features or small artifacts. In this example, there are two circles of diameters 80 and 60 and the separation distance (void region) is varied from 1.0 to 20.0 in increments of 1.9. The resolution is 1 pixel per unit length. Given this configuration, we would expect to resolve separation distances down to about 2.0 based mainly on the resolution of the image.

A plot of the measured vs. actual value of the separation distance is shown below. This plot is almost exactly linear for separation distances down to about 5.0 below which the measured value is a constant. In the output, the relative error actually starts to become larger than expected when the separation distance is 6.7.

This could suggest some room for improvement to imageruler's algorithm.

notebook

import sys
sys.path.append("../imageruler")

import imageruler
from matplotlib import pyplot as plt
import numpy as np
from regular_shapes import disc

def separated_discs(
    left_diameter: float, 
    right_diameter: float, 
    separation_distance: float
) -> np.ndarray:
    left_center = (0, -0.5 * (left_diameter + separation_distance))
    right_center = (0, 0.5 * (right_diameter + separation_distance))

    left_disc = disc(resolution, phys_size, left_diameter, left_center)
    right_disc = disc(resolution, phys_size, right_diameter, right_center)

    return left_disc ^ right_disc

resolution = 1  # number of pixels per unit length
phys_size = (130, 200)  # physical size of the entire image

left_diameter = 80
right_diameter = 60
separation_distance = 3
image = separated_discs(left_diameter, right_diameter, separation_distance)

measured_separation_distance = imageruler.minimum_length_void(image) 
print(f"Measured separation distance: {measured_separation_distance:.6f}")
print(f"Actual separation distance:  {separation_distance:.6f}")

Measured separation distance: 4.401367
Actual separation distance:  3.000000

fig, ax = plt.subplots()
ax.imshow(image)
fig.savefig('separated_discs.png', dpi=150, bbox_inches='tight')

min_separation_distance = 1.0
max_separation_distance = 20.0
num_separation_distance = 11
separation_distances = np.linspace(min_separation_distance, max_separation_distance, num_separation_distance)
measured_separation_distance = np.zeros(num_separation_distance)

delta_separation_distance = (max_separation_distance - min_separation_distance) / (num_separation_distance - 1)
pixel_length = 1 / resolution
print(f"Δd = {delta_separation_distance:.6f}, Δp = {pixel_length:.6f}")

for i, separation_distance in enumerate(separation_distances):
    image = separated_discs(left_diameter, right_diameter, separation_distance)
    measured_separation_distance[i] = imageruler.minimum_length_void(image)
    err = abs(measured_separation_distance[i] - separation_distance) / separation_distance
    print(f"separation_distance:, {separation_distance:.6f}, {measured_separation_distance[i]:.6f}, {err:.6f}")

Δd = 1.900000, Δp = 1.000000
separation_distance:, 1.000000, 4.149414, 3.149414
separation_distance:, 2.900000, 4.401367, 0.517713
separation_distance:, 4.800000, 4.401367, 0.083049
separation_distance:, 6.700000, 6.416992, 0.042240
separation_distance:, 8.600000, 8.432617, 0.019463
separation_distance:, 10.500000, 10.448242, 0.004929
separation_distance:, 12.400000, 12.463867, 0.005151
separation_distance:, 14.300000, 14.479492, 0.012552
separation_distance:, 16.200000, 16.495117, 0.018217
separation_distance:, 18.100000, 18.510742, 0.022693
separation_distance:, 20.000000, 20.526367, 0.026318

fig, ax = plt.subplots()
ax.plot(separation_distances, measured_separation_distance, 'bo')
ax.plot(separation_distances, separation_distances, 'k-')
ax.set_xlabel('actual separation distance')
ax.set_ylabel('measured separation distance')

fig.savefig('separation_distance_measured_vs_actual.png', dpi=150, bbox_inches='tight')

[stevengj]

This example is showing a ≈ 50% error when the separation is a little under 3 pixels, about a 1.5-pixel error bar. This seems pretty good to me?

Recall that we only count "interior" pixels when looking at the difference of morphological transforms, in order exclude discretization artifacts. This seems like it could give error bars of up to 2 pixels, since an interior pixel needs to have one pixel on either side of it. Maybe there is a different way to define "interior" so that something 2 pixels wide is counted as having 1 "interior pixel", that would lower the error bar.

Note also that just in your construction of the image you could have an error of about 1 pixel in the separation just from the discretization process. (e.g. you could have a separation that is supposed to be 1.99 pixels, but the boundaries fall just past the centers of two adjacent pixels so it gets increased to 3 pixels.) So you really have 2 sources of error here, and they are additive.

[stevengj]

cc @mawc2019

[mawc2019]

Recall that we only count "interior" pixels when looking at the difference of morphological transforms, in order exclude discretization artifacts. This seems like it could give error bars of up to 2 pixels, since an interior pixel needs to have one pixel on either side of it.

Yes. Consider we strip off the interface solid pixels from a solid pattern. After this operation, if there is still at least one pixel left, the original solid pattern should span at least 3 pixels in both x and y directions. Therefore, if the difference between opening and closing is a thin pattern (at most 2-pixel wide), such difference will not be counted as a sign of lengthscale violation.

…… down to about 5.0 below which the measured value is a constant. In the output, the relative error actually starts to become larger than expected when the separation distance is 6.7.

At small lengthscale, the difference between opening and closing can be a thin pattern even if violation attains. Therefore, the lengthscale tends to be overestimated in this regime.

This could suggest some room for improvement to imageruler's algorithm.

Yes. For example, we might introduce another violation criterion without stripping off interface pixels: if a 2×2 solid pattern is formed after image subtraction, i.e., $\rho-\mathcal{O}(\rho)$, $\mathcal{C}(\rho) - \rho$, and $\mathcal{C}(\rho)-\mathcal{O}(\rho)$ , lengthscale violation is counted. I do not know if this criterion can replace the current one. If the answer is no, we might need use the two criteria together, which increases the computational cost.

Note also that just in your construction of the image you could have an error of about 1 pixel in the separation just from the discretization process.

Yes, but this issue cannot be resolved by introducing a new criterion. Perhaps we can improve the construction accuracy a little bit by specifying whether the 2d input array is defined at pixel centers of pixel vertices.

[mfschubert]

The needed improvement here likely relates to the manner in which "ignored violations" are calculated. Currently, the algorithm ignores all border features, even if the feature is one or two pixels wide. It is probably better in general to only ignore borders for large features.

I have a version of this implemented here: https://github.com/mfschubert/topology/blob/main/src/tometrics/metrics.py#L371

[stevengj]

Closed by #30