Using own dataset already uploaded into jupyter notebook

[wolfman30]

I'm using a nuclei dataset from Broad Institute in the Kaggle 2018 Data Science Bowl.. I already have 670 training images and masks for each nucleus uploaded on my jupyter notebook. I understand how to subclass the configuration part, but not the dataset. I don't know how to use these three methods within the new subclass NucleiDataset(utils.Dataset) : (1) def load_nuclei(self), (2) def load_mask(self, image_id), and (3) image_reference(self, image_id).

Here's the code I already used to prepare the dataset for a U-Net:

`IMG_WIDTH = 256 IMG_HEIGHT = 256 IMG_CHANNELS = 3 TRAIN_PATH = 'train/' TEST_PATH = 'test/'

train_ids = next(os.walk(TRAIN_PATH))[1] test_ids = next(os.walk(TEST_PATH))[1]

X_train = np.zeros((len(train_ids), IMG_HEIGHT, IMG_WIDTH, 3), dtype=np.uint8) Y_train = np.zeros((len(train_ids), IMG_HEIGHT, IMG_WIDTH, 1), dtype=np.bool)

for n, id_ in tqdm(enumerate(train_ids), total=len(train_ids)): path = TRAINPATH + id img = imread(path + '/images/' + id_ + '.png')[:,:,:3] img = resize(img, (IMG_HEIGHT, IMG_WIDTH), mode='constant', preserve_range=True) X_train[n] = img mask = np.zeros((IMG_HEIGHT, IMG_WIDTH, 1), dtype=np.bool)

gets and resizes the masks that contain the individual segmented nuclei

#the targets for the model
for mask_file in next(os.walk(path + '/masks/'))[2]:
    mask_ = imread(path + '/masks/' + mask_file)
    mask_ = np.expand_dims(resize(mask_, (256, 256), mode='constant', 
                                  preserve_range=True), axis=-1)
    mask = np.maximum(mask, mask_)
Y_train[n] = mask

X_test = np.zeros((len(test_ids), IMG_HEIGHT, IMG_WIDTH, 3), dtype=np.uint8) sizestest = [] for n, id in tqdm(enumerate(test_ids), total=len(test_ids)): path = TESTPATH + id img = imread(path + '/images/' + id_ + '.png')[:,:,:IMG_CHANNELS] sizes_test.append([img.shape[0], img.shape[1]]) img = resize(img, (IMG_HEIGHT, IMG_WIDTH), mode='constant', preserve_range=True) X_test[n] = img

from sklearn.model_selection import train_test_split x_train, x_valid, y_train, y_valid = train_test_split(X_train, Y_train, test_size=0.1, random_state=8011)`

So I tried to incorporate this code into those three functions, but I'm not sure how to correctly do so. The training data has the following file to each image in the training set: 'train/image_id/images/image_id.png'. Here's the path to the masks: 'train/image_id/masks/image_id.png. Maybe important to note that there are several masks for each individual image as each mask is a single segmented nucleus without overlap with the other masks.

Maybe I'm overcomplicating this, because it seems that the current setup is expecting a user's own dataset to be more steps away as downloadable from an external website, but in my case, the entire dataset is already uploaded and unzipped to my jupyter notebook.

Would anyone be willing and able to guide me on how to subclass Dataset to fit the already uploaded dataset to this model?

[wolfman30]

I advanced to a new issue.

[hathemi]

any advice here please?