training data synthesis issue

[suvi-coder]

as per instructions i have 4kx4k images in folder. i don't see any errors or results when i run bash draw_fake.sh . detailed instructions on synthesizing dataset will be appreciated. thanks

[scotthoule]

Hi suvi-coder,

Is there anything in nohup.out? Can you confirm if there's any python program running on background by running "ps x | grep draw_fake".

Thx

[suvi-coder]

Hi scotthoule, Thanks for the quick response. nohup.out is empty and when i run ps x | grep draw_fake ,got this 60 tty1 S 0:00 grep --color=auto draw_fake.

[scotthoule]

That's weird. So there are no images under output/image/?

Can you run the following: python draw_fake.py 0 1

Maybe the error should pop out this way.

[suvi-coder]

Hi , I want to generate just mask images. How can generate and save them. Thank you

[scotthoule]

Hi there,

Maybe you can change replace file https://github.com/SBU-BMI/quip_cnn_segmentation/blob/555bbf20b612361f63d4d5f6585a387bacfa83aa/training-data-synthesis/nuclei_synthesis_40X_online/image_synthesize.py with the following code. But be aware, the density of masked nuclei is not derived according to some real textures, so you might lose some diversity of generated masks.

import numpy as np
import pickle
import png
from PIL import Image, ImageDraw
from polygon import generatePolygon
from mask2image_otsu import Mask2Image
from scipy import ndimage
from sys import stdout

# Synthesize patches of 460x460 pixels first,
# then cut the surrounding 30 pixels off to get patches of 400x400 pixels.
size = 460;
cm = 30;

# Apply a QUAD transform with quad_marg.
# This transform models the correlation of shape and size of nearby nuclei.
quad_marg = 310;

# This function returns a set of parameters for nuclear polygon generation.
def get_rand_polygon_param(s):
    x, y = int(np.random.rand()*s), int(np.random.rand()*s);
    rad = np.random.rand()*4.5 + 8.5;
    irr = 0.6+np.random.rand()*2.0;
    spike = 0.04+np.random.rand()*0.20;
    nverti = int(np.random.rand()*8+10);
    return x, y, rad, irr, spike, nverti;

# This function returns a set of parameters for noise polygon generation.
def get_rand_noise(s):
    x, y = int(np.random.rand()*s), int(np.random.rand()*s);
    rad = np.random.rand()*5.0+6.0;
    irr = 40.0;
    spike = 40.0;
    nverti = 40;
    return x, y, rad, irr, spike, nverti;

# Draw polygon mask according to polygon parameters
def draw_polygon(x, y, rad, irr, spike, nverti, s):
    vertices = generatePolygon(x, y, rad, irr, spike, nverti);
    mask = Image.fromarray(np.zeros((s, s, 3), dtype=np.uint8));
    draw = ImageDraw.Draw(mask);
    draw.polygon(vertices, fill=(1,1,1));
    return np.array(mask);

# Apply a random QUAD transformation.
# This transform models the correlation of shape and size of nearby nuclei.
def random_transform(mask, s, q):
    return np.array(Image.fromarray(mask).transform((s, s), Image.QUAD, q, Image.NEAREST));

# Generate a random QUAD transformation.
def rand_quad_trans():
    q = (np.random.rand()*quad_marg,
            np.random.rand()*quad_marg,
            np.random.rand()*quad_marg,
            size+quad_marg-np.random.rand()*quad_marg,
            size+quad_marg-np.random.rand()*quad_marg,
            size+quad_marg-np.random.rand()*quad_marg,
            size+quad_marg-np.random.rand()*quad_marg,
            np.random.rand()*quad_marg);
    return q;

# Generate a new nucleus with mask, centroid (detect), and contour
def rand_nucleus(nsize, qm, q, nsize_bias):
    mask = np.array([-1]);
    ntry = 0;
    while (mask>0).sum() <= 140 and (ntry < 500):
        x, y, rad, irr, spike, nverti = get_rand_polygon_param(nsize+qm);
        mask = draw_polygon(x, y, int(nsize_bias*rad), irr, spike, nverti, nsize+qm);
        mask = random_transform(mask, nsize, q);
        ntry += 1;
    detect = np.zeros((mask.shape[0], mask.shape[1], 3), dtype=np.uint8);
    x, y, _ = np.nonzero(mask);
    detect[(np.min(x)+np.max(x))//2, (np.min(y)+np.max(y))//2, :] = 255;

    dx = ndimage.sobel(mask[:,:,0], 0);
    dy = ndimage.sobel(mask[:,:,0], 1);
    contour = ((dx>0) + (dy>0))[:,:,np.newaxis] * (mask==0);

    return mask, detect, contour;

# Generate a clutered noise mask
def get_noise_mask(nsize, qm, star_noise):
    mask = np.array([-1]);
    ntry = 0;
    quad = rand_quad_trans();
    while np.sum(mask > 0) <= 10 and (ntry < 100):
        if star_noise:
            x, y, rad, irr, spike, nverti = get_rand_noise(nsize+qm);
        else:
            x, y, rad, irr, spike, nverti = get_rand_polygon_param(nsize+qm);
        mask = draw_polygon(x, y, rad, irr, spike, nverti, nsize+qm);
        mask = random_transform(mask, nsize, quad);
        ntry += 1;
    return mask;

# The main API of this file.
# Return a fake image patch with nuclear mask and related info.
def get_new_fake_image():
    # First, get some information of real texture
    dense_lvl = 2

    # Generate the number of nuclei, according to the information of real texture.
    if np.random.rand() < 0.10 and dense_lvl <= 1:
        nuclei_num = 0;
    else:
        nuclei_num = np.random.randn()*40 + dense_lvl*30 + 20.0;
    if nuclei_num < 0:
        nuclei_num = 0;
    if nuclei_num > 256:
        nuclei_num = 256;
    nuclei_num = int(nuclei_num);
    # Random number of noise type 1
    noise_num = np.random.randint(low=0, high=100);
    # Random number of noise type 2
    noise2_num = np.random.randint(low=0, high=5);
    # Size bias of all nuclei
    nsize_bias = np.random.randn()*0.32 + dense_lvl/8.0 + 1.2;

    # mask, centroid (detect), and contour are accumulative
    mask = np.zeros((size, size, 3), dtype=np.uint32);
    detect = np.zeros((size, size, 3), dtype=np.uint32);
    contour = np.zeros((size, size, 3), dtype=np.uint32);
    quad = rand_quad_trans();
    average_size = 0.0;
    for nuclei_no in range(nuclei_num):
        # return a mask to add on
        im_add, det_add, con_add = rand_nucleus(size, quad_marg, quad, nsize_bias);
        overlap = np.sum((im_add * mask)>0) / np.sum(im_add>0).astype(np.float32);
        ntry = 0;
        # if the overlapping of nuclei is too large, return a new mask
        while (overlap > 0.15) and (ntry < 50):
            im_add, det_add, con_add = rand_nucleus(size, quad_marg, quad, nsize_bias);
            overlap = np.sum((im_add * mask)>0) / np.sum(im_add>0).astype(np.float32);
            ntry += 1;
        mask += im_add;
        detect += det_add;
        contour += con_add;
        average_size += np.sum(mask[:,:,0]>0);
    if nuclei_num > 0:
        average_size /= nuclei_num;

    noise_mask = np.zeros((size, size, 3), dtype=np.uint8);
    for noise_no in range(noise_num):
        noise_mask += get_noise_mask(size, quad_marg, star_noise=True);

    noise2_mask = np.zeros((size, size, 3), dtype=np.uint8);
    for noise2_no in range(noise2_num):
        noise2_mask += get_noise_mask(size, quad_marg, star_noise=False);

    # Convert mask to image and save the image
    # This is achieved by painting textures inside and outside the mask
    image, refer, source, nucl, cyto, intp_nucl_mask = (
        mask * 100, mask * 100, mask * 100, mask * 100, mask * 100, mask * 100)

    return image[cm:-cm, cm:-cm, :], mask[cm:-cm, cm:-cm, :], detect[cm:-cm, cm:-cm, :], \
           contour[cm:-cm, cm:-cm, :], refer[cm:-cm, cm:-cm, :], \
           source[cm:-cm, cm:-cm, :], nucl[cm:-cm, cm:-cm, :], \
           cyto[cm:-cm, cm:-cm, :], intp_nucl_mask[cm:-cm, cm:-cm, :], '';

[suvi-coder]

Hello scotthoule, Thanks for the code. Yes, like you said its failing to derive density of nuclei .

Why am i not getting any results neither initial synthetic images nor mask images. I am using 2000 X 2000 size (141 in counts) images. draw_real.py is working. when i run draw_fake.py 0 1 or 0 5 i am not getting any results.

Thank you

[scotthoule]

Hi,

Can you add some logs some where in the code, to get an idea of where it's getting stuck at? I would recommend adding print before these lines: https://github.com/SBU-BMI/quip_cnn_segmentation/blob/master/training-data-synthesis/nuclei_synthesis_40X_online/image_synthesize.py#L21 https://github.com/SBU-BMI/quip_cnn_segmentation/blob/master/training-data-synthesis/nuclei_synthesis_40X_online/image_synthesize.py#L122 https://github.com/SBU-BMI/quip_cnn_segmentation/blob/master/training-data-synthesis/nuclei_synthesis_40X_online/image_synthesize.py#L134 https://github.com/SBU-BMI/quip_cnn_segmentation/blob/master/training-data-synthesis/nuclei_synthesis_40X_online/image_synthesize.py#L138 https://github.com/SBU-BMI/quip_cnn_segmentation/blob/master/training-data-synthesis/nuclei_synthesis_40X_online/image_synthesize.py#L153 https://github.com/SBU-BMI/quip_cnn_segmentation/blob/master/training-data-synthesis/nuclei_synthesis_40X_online/image_synthesize.py#L160

https://github.com/SBU-BMI/quip_cnn_segmentation/blob/master/training-data-synthesis/nuclei_synthesis_40X_online/mask2image_otsu.py#L265 https://github.com/SBU-BMI/quip_cnn_segmentation/blob/master/training-data-synthesis/nuclei_synthesis_40X_online/mask2image_otsu.py#L282 https://github.com/SBU-BMI/quip_cnn_segmentation/blob/master/training-data-synthesis/nuclei_synthesis_40X_online/mask2image_otsu.py#L291 https://github.com/SBU-BMI/quip_cnn_segmentation/blob/master/training-data-synthesis/nuclei_synthesis_40X_online/mask2image_otsu.py#L300

https://github.com/SBU-BMI/quip_cnn_segmentation/blob/master/training-data-synthesis/nuclei_synthesis_40X_online/mask2image_otsu.py#L143 https://github.com/SBU-BMI/quip_cnn_segmentation/blob/master/training-data-synthesis/nuclei_synthesis_40X_online/mask2image_otsu.py#L151

[suvi-coder]

i did try printing few things before too. when we print (m2image) should we expect anything else than this <nuclei_synthesis_40X_online.mask2image_otsu.Mask2Image object at 0x000001DD80701F08>. Thank you

[scotthoule]

That looks fine. m2image is indeed a Mask2Image object.

[suvi-coder]

sorry for troubling you so much. But, Thats all i get when i run python draw_fake.py 0 1 and nothing else. I don't know where am i going wrong.

[scotthoule]

Hi suvi-coder,

No worries. Did you add a print after each line of code I suggested before, and what is the log you see?

[suvi-coder]

it shows tthe below error at line 133 quip_cnn_segmentation-master\training-data-synthesis\nuclei_synthesis_40X_online\image_synthesize.py", line 133 print(im_add,det_add,con_add) ^ TabError: inconsistent use of tabs and spaces in indentation

[scotthoule]

Hi suvi-coder,

I used spaces for python indentation. Make sure you also use spaces, not tabs, before print.

[jingzi97]

Hi scotthoule, I also encountered the same problem as the friend above ，run python draw_fake.py 0 1 and nothing else.I found draw_fake.py 15line get_new_fake_image() can't pass data to the previous variable .I want to confirm one thing ，whether only need to put real pathological images in the training-data-synthesis/nuclei_synthesis_40X_online/real_tiles folder .I'm so sorry to interrupt ,I really tried for a long time without finding the reason ，Hope to hear from you ！

[jingzi97]

mask2image_otsu.py/def draw_random_texture_try/hed_mean is always around -0.5 +，can't break out of this loop if hed_mean < -1.33 or hed_mean > -1.1: return False, False, False, False, False, False, False, False, False;

[scotthoule]

Hi jingzi97,

Thanks for reaching out! Based on your comment, the color of your real images is a bit unexpected. Can you enlarge the range of [-1.33, -1.1] maybe to [-10.0, 0.0] and try again? If you run into other problems or the synthetic images look weird, let me know.

Thx

[jingzi97]

Hi scotthoule, i have found the reason why draw_fake.py can't run , because make2image_otsu.py/def sampe_xy_fxfy return need add Spaces. I have another question to consult ，the data pixel I found is 460×700 ,so ,the code where i need to modify. I really appreciate your help.

[jingzi97]

Hi scotthoule, I have put 1000×1000 images in real_tiles folder,but always output the following error : only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices

[scotthoule]

Hi,

Sorry for the bug. Which line of code were you getting the error from?