Model training

[umang4002]

In the model training in the 16th line

model.fit(x=image_augmented, y=mask_augmented, epochs=1, batch_size=4, validation_data=(x_valid, y_valid), verbose=1, callbacks=[csv_logger])

_NotFoundError: ProgressFull/kvasir_progress_csv_DuckNet_filters_172023-12-01 20:02:22.394228.csv; No such file or directory. Error is coming. I am using KVASIR dataset for training. Please help

[maenstru56]

Hello,

GitHub doesn't allow empty directories to be added to repositories so that's why we couldn't add the whole directory structure for our project. You can either create the "ProgressFull" directory and "ModelSaveTensorFlow" directory with its sub-directories manually, or modify the code to check if they exist and if not to create them automatically. As a guide, your directory structure should look like this: | CustomLayers | ImageLoader | ModelArchitecture | ModelSaveTensorFlow | | kvasir | | cvc-clinicdb | | cvc-colondb | | etis-laribpolypdb | | ADD ANY OTHER DATASET DIRECTORY HERE, MAKE SURE IT HAS THE SAME NAME AS THE ONE IN THE DATA LOADER AND IN THE MODEL NOTEBOOK ("dataset_type" variable) | ProgressFull

Additionally, if you want to train the model on different datasets, I suggest that you first run the training for 1-2 epochs, force the model to save (set "min_loss_for_saving = 1.0") and check that the model was saved properly, and that it generated the progress files.

I hope this helps!

[umang4002]

Thankyou for the information. I will surely give it a try.

[umang4002]

Thankyou for your help, I followed the steps suggested by you, but still same error persists. Can you please help me out ?

[maenstru56]

Hello,

I think the most likely explanation for the error is the way the save paths are defined. Based on your operating system you might want to modify these lines of code: ct = datetime.now()

model_type = "DuckNet"

progresspath = 'ProgressFull/' + dataset_type + '_progress_csv_' + model_type + '_filters_' + str(filters) + '\' + str(ct) + '.csv' progressfullpath = 'ProgressFull/' + dataset_type + '_progress_' + model_type + '_filters_' + str(filters) + '\' + str(ct) + '.txt' plotpath = 'ProgressFull/' + dataset_type + '_progress_plot_' + model_type + '_filters_' + str(filters) + '\' + str(ct) + '.png' modelpath = 'ModelSaveTensorFlow/' + dataset_type + '/' + model_type + '_filters_' + str(filters) + '\' + str(ct)

I suggest that you try experimenting with the following:

• First, try replacing the character ":" from the datetime with a "_", as in some cases this may be problematic. I suggest the following modification: progresspath = './ProgressFull/' + dataset_type + '_progress_csv_' + model_type + '_filters_' + str(filters) + '_' + str(ct).replace(':', '\') + '.csv' progressfullpath = './ProgressFull/' + dataset_type + '_progress_' + model_type + '_filters_' + str(filters) + '_' + str(ct).replace(':', '\') + '.txt' plotpath = './ProgressFull/' + dataset_type + '_progress_plot_' + model_type + '_filters_' + str(filters) + '_' + str(ct).replace(':', '\') + '.png' modelpath = './ModelSaveTensorFlow/' + dataset_type + '/' + model_type + '_filters_' + str(filters) + '_' + str(ct).replace(':', '\')

• If the above solution doesn't work, try to change "\/" into "\" or "\\", as some operating systems prefer the latter

• Finally, if nothing else seem to work, use the "os" python library for handling the paths, you can find some online documentation on how to use it here: https://docs.python.org/3/library/os.html