Closed nightandweather closed 3 years ago
Hi @nightandweather,
It seems that the issue you are experiencing is that crop_single_mammogram
function from src/cropping/crop_single.py
does not remove background for some of your images. Is this correct?
Our cropping algorithm has some strict assumptions about the data. If your dataset does not satisfy any of these assumptions, the cropping algorithm will not work well:
For more information, please refer to Algorithm 1 in our data report.
I would recommend trying any one of the following to address the issue:
img_mask = img > 0
, try img_mask = img > thres
where thres is some small positive integer. Try multiple different values until it works well.Thank you for your comment! Just like you said, image threshold method works!
Actually.. there are more issues to adapt this github code in our dataset
Attached ROC curve is result of following your Github Repository.
Use 10 malignant annotated patients(single breast diagnosed malignant,left, right) * 4 mammography standard view to get ROC curve, and annotation label list is
[1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0]
ex)If the left breast is malignant, it is assumed that it is malignant in the l-mlo,l-cc view.
and model got low score
Maybe I made a mistake in preprocessing, As stated in the paper, the pixel array of dcm was saved as png as uint 16 data using the standardized code provided in github. this is dicom to 16 bit png code
`# 아무래도 16비트 변환이 안된 모양이다. 다시 만들어보자 """
cv2.imread default가 8bit로 읽어들인다.
""" ######################################################################### import natsort import imageio import numpy as np def standard_normalize_single_image(image): """ Standardizes an image in-place """
image =image - np.mean(image) # np.mean을 통과하면 float64로 변하는 모양인데..
image /= np.maximum(np.std(image), 10**(-5))
return image
def load_dcm_data(path): shpfiles = [] labels = []
for dirpath, subdirs, files in os.walk(path):
for x in files:
if x.endswith(".dcm"):
shpfiles.append(os.path.join(dirpath, x))
return natsort.natsorted(shpfiles)
#######################################################################################
import pydicom
import matplotlib.pyplot as plt
from sklearn.preprocessing import MinMaxScaler
base_malignant_folder = r'/home/ncc/Desktop/2020_deep_learning_breastcancer/submit_breast_SN/malignant'
test_folder_path = r'/home/ncc/Desktop/2020_deep_learning_breastcancer/test_malignant_folder/'
sample_malignant_list = load_dcm_data(base_malignant_folder)[0:40]
sample_malignant_list
threshold = 100
for f in sample_malignant_list:
file_name = os.path.basename(f)
ds = pydicom.dcmread(f) # read dicom image
img = ds.pixel_array # get image array (0~ 4095)
img[img<150]=0
plt.hist(img.ravel());
plt.show()
#논문에서 원하는건 16bit의 standardized된 mammmography png임
img = standard_normalize_single_image(img)
img2 = (65535*(img - img.min())/img.ptp()).astype(np.uint16) # (0~65535)
# img_threshold = img_threshold.reshape(img2.shape)
# Get the information of the incoming image type
print('scaler적용',img2.max(),img2.min())
imageio.imwrite(test_folder_path + file_name.split('.')[0]+'.png',img2.astype(np.uint16)) # write png image`
And while I was looking at the last issues, https://github.com/nyukat/breast_cancer_classifier/issues/9#issuecomment-489370370
Do I have to change the annotated label according to the results of the model in order to raise the AUC core?
and, I'd like to know the value of the parameter to get the 3ch(gray, benign, malignant) image in the paper(image-heatmap model parameter). I made various parameter values through dictionaries and carried out grid searching, but no satisfactory figure was found.
Thank you for accepting my question!
It seems that you are trying to load the images and feed it to the model on your own, instead of using our pipeline. I noticed some misunderstandings and problematic lines of code in your custom pipeline.
img[img<150]=0
is problematic because you are modifying the image directly, treating all values below 150 as background. This might or might not be a good threshold for the background, but even if it is, it could also erase some regions inside the breast. In addition, even if this thresholding works reasonably for some of your images, it might not be an ideal value for the other images. What I meant is to use thresholding when generating mask in the cropping algorithm. standard_normalize_single_image
function does not return anything as output, but rather modifies the given array in-place. Therefore, if you did not change our function, img = standard_normalize_single_image(img)
will assign img as None. It seems that you changed the behavior of our function so that there is some return value. This might be problematic later when some of our code calls this function.cv2.imread
will read the saved 16 bit png files as 8 bit images. You should be using the provided read_image_png
function, which uses imageio.imread
function instead.As @kjgeras mentioned in https://github.com/nyukat/breast_cancer_classifier/issues/9#issuecomment-490127933, even the slightest differences in the loading pipeline can lead to random predictions.
Even if all the preprocessing is done correctly, if your dataset itself is drastically different from our own, it could also affect the performances as mentioned in #19.
I suggest you look at the debugging strategy written by @kjgeras which can be found in https://github.com/nyukat/breast_cancer_classifier/issues/9#issuecomment-490146418
At this stage, I recommend that you clone our repository again without any modification, fix the cropping algorithm to use nonzero masking threshold, and use the provided pipeline as-is (run.sh
or run_single.sh
). For the images, you can just save dicom pixel_array as 16-bit png files without any standardization or normalization. This way, you can be more sure that you are preprocessing the images the way we expect. If you still do not get a reasonable performance, you can try examining the dicom metadata to see if you applied the same filtering criteria as we did.
And while I was looking at the last issues, #9 (comment)
Do I have to change the annotated label according to the results of the model in order to raise the AUC core?
No, you must not change the label according to the results of the model when calculating AUC. What the comment was discussing is how to set the decision threshold, which has nothing to do with AUC calculation. On the other hand, AUC is the metric that measures the model's ability to distinguish between malignant and not malignant cases at all threshold values. To learn more about AUC, I recommend reading this article about receiver operating characteristic curve.
and, I'd like to know the value of the parameter to get the 3ch(gray, benign, malignant) image in the paper(image-heatmap model parameter). I made various parameter values through dictionaries and carried out grid searching, but no satisfactory figure was found.
I am not sure what you mean by this sentence. Do you mean you changed heatmap parameters such as stride_fixed
and patch_size
from src/heatmaps/run_producer.py or src/heatmaps/run_producer_single.py? You should not have changed these parameters, as the classifier model expects heatmaps generated with predefined parameters we provided.
The reason why you are not be seeing satisfactory heatmaps might also be related to the differences in datasets and issues in your custom pipeline I explained in https://github.com/nyukat/breast_cancer_classifier/issues/42#issuecomment-734422874 . If you are not feeding the images the way we do, you cannot expect reasonable heatmaps.
thank you for your work sharing, I'm trying to adapt your repository to our dataset.
`score_heat_list = [] import glob def make_dir(name): if not os.path.isdir(name): os.makedirs(name) print(name, "폴더가 생성되었습니다.") else: print("해당 폴더가 이미 존재합니다.")
make_dir('save_imageheatmap_model_figure_folder')
def json_extract_feature(json_data): patient=json_data['case_id']
from skimage import draw def polygon2mask(image_shape, polygon): """Compute a mask from polygon. Parameters
############################################################################################################################## from tqdm import tqdm from src.heatmaps.run_producer_single import produce_heatmaps import json from PIL import Image annotation_folder = r'/home/ncc/Desktop/2020_deep_learning_breastcancer/annotation_SN/' import pickle for png in tqdm(png_list[0:8]): print(PATH+png) crop_single_mammogram(PATH+png, horizontalflip = 'NO', view = png.split('')[1].split('.')[0], cropped_mammogram_path = PATH+'cropped_image/'+png, metadata_path = PATH+png.split('.')[0]+'.pkl',num_iterations = 100, buffer_size = 50) print(PATH+'cropped_image/'+png) get_optimal_center_single(PATH+'cropped_image/'+png,PATH+png.split('.')[0]+'.pkl') model_input = load_inputs( image_path=PATH+'cropped_image/'+png, metadata_path=PATH+png.split('.')[0]+'.pkl', use_heatmaps=False, )
####################################################################################################################################
parameters = dict( device_type='gpu', gpu_number='0',
) ###########################################################################################################################
read annotation SN00000016_L-CC.png
코드를 읽어보면 이름이 같은 JSON 파일을 4번 읽어오고 있음.. 코드 경량화때 해결 필요
annotation 기준은 CROP된 이미지가 아니라, 원본 이미지임, 그런데 이미지로 보여주는건 CROP된 이미지로 보여주고 있음..
########################################################################################################################### random_number_generator = np.random.RandomState(shared_parameters["seed"])
Attached file is cropped mammography which is made by this code. Issue is some mammogram doesn't crop well. Am I doing something wrong?