Issue in the code & Cannot replicate the accuracy levels given in the paper

[artiransaygin]

Hi,

I was going over the code and realized a questionable line. When you are fitting the model using "model.fit" command, you are giving the entire data which contains the test data. I don't understand what's happening there. Regardless of that, I cannot get the accuracy levels presented in the paper for the "Sanitation" dataset. When I train the model as it is in the code I get 86.77% and when I train the model using the training data only I get 82.3%. I think the model parameters match the ones given in the paper. Can you explain this situation? Thanks!

[zhangzhao156]

Thank you for your feedbacks, model.fit has the arg 'validation_split=0.3', the model still trians on the training dataset. The uploaded codes are reorganized from the codes we actually run, which are for reference only. The accuracies on the paper was obtained by executing many times and getting the highest one. In addtion, the sanitation is the dataset we collect, you can just focus on your datasets.

[artiransaygin]

Please correct me if I am wrong but randomly splitting the entire data into training and validation sets do not guarantee a fair test step since test data might exist in the training and validation data. Secondly, I want to compare your model against my model, so I don't want U-Net to perform worse than it evidently can (as backed up by your paper) because in that case, it wouldn't be an ideal comparison.

[zhangzhao156]

Thank your suggestion. I think you are right, it's better to write it as "history=model.fit(X_train, y_train, batch_size=batch_size, epochs=epochs, validation_data=(X_test, y_test), callbacks=callbacks) ". Besides, you can adjust the learning rate or the other settings to get a better result.