Mask not working

[glopezzz]

Hi,

I'm working on a Kaggle proyect: https://www.kaggle.com/c/hpa-single-cell-image-classification

When I load the test data and apply the segmentator.pred_cells() function, it all works ok.

But when I apply thelabel_cell() function, the array returned is empty (all 0's):

The function where I load the images from Kaggle is:

def load_images(df : pd.DataFrame, root='../input/hpa-single-cell-image-classification/test/'):
    blue = []
    rgb = []
    blue_scaled = []
    rgb_scaled = []
    for i, row in df.iterrows():
        r = os.path.join(root, f'{row.ID}_red.png')
        y = os.path.join(root, f'{row.ID}_yellow.png')
        b = os.path.join(root, f'{row.ID}_blue.png')
        r = cv2.imread(r, cv2.IMREAD_GRAYSCALE)
        y = cv2.imread(y, cv2.IMREAD_GRAYSCALE)
        b = cv2.imread(b, cv2.IMREAD_GRAYSCALE)
        blue_image = cv2.resize(b, (512, 512))
        rgb_image = cv2.resize(np.stack((r, y, b), axis=2), (512, 512))
        blue.append(blue_image)
        blue_scaled.append(blue_image/255.)
        rgb.append(rgb_image)
        rgb_scaled.append(rgb_image/255.)

    return blue, rgb, blue_scaled, rgb_scaled

Cell Segmentation plt.imshow(cell_segmentations[0])

NUC_MODEL = "../input/hpacellsegmodel/cell-model.pth"
CELL_MODEL = "../input/hpacellsegmodel/nuclei-model.pth"
segmentator = cellsegmentator.CellSegmentator(
    NUC_MODEL,
    CELL_MODEL,
    padding=True,
    device="cuda",
    multi_channel_model=True,
)

I'll leave the code here, although the same code has worked with the test data...

batch_size = 24

for i in tq.tqdm(range(0, data_size, batch_size), position=0, leave=True):

    start = i
    end = min(len(data_df), start + batch_size)
    _, _, blue_batch, rgb_batch = load_images(data_df[start:end])
    nuc_segmentations = segmentator.pred_nuclei(blue_batch)
    cell_segmentations = segmentator.pred_cells(rgb_batch, precombined=True)
    j=0
    for data_id, nuc_seg, cell_seg in zip(data_df.ID.to_list()[start:end], nuc_segmentations, cell_segmentations):
        _, image_mask = utils.label_cell(nuc_seg, cell_seg)

        pred_string = ''
        numbers = list(np.unique(image_mask)[1:])
        f, ax = plt.subplots(1, 3, figsize=(16,16))
        ax[0].imshow(nuc_seg)
        ax[1].imshow(cell_seg)
        ax[2].imshow(image_mask)
        plt.show()

The outputs are:

Also, in the images where it does capture cell masks, it ignores plenty of the cells...

Any idea why this function doesn't work?

Thank you in advance! Escanor

[cwinsnes]

The code itself looks like it should be correct. How do you create your segmentator object? Many problems seems to be solved if you set padding=True during instantiation.

[glopezzz]

Hi @cwinsnes

I tried just that and it still doesn't work... What could it be?

The weirdest part is that the segmentation works... I don't get it

[cwinsnes]

The test images in the Kaggle set are 16bit while the train ones are 8bit, as discussed in https://www.kaggle.com/c/hpa-single-cell-image-classification/discussion/223745

As the profile images and names are alike, I assume the cv2 solution helped you there?

[glopezzz]

Hi @cwinsnes

I've been having issues with this function again. I updated the issue above with more proofs.

As you can see, I'm using both padding=True and cv2.imread(b, cv2.IMREAD_GRAYSCALE)

I have no clue what might be going wrong, but I don't see many people having this sort of truble, which is giving me quite a headache

The full notebook is here, in case you want to give it a look.

Thank you in advance :)