search

aj1365 / ResUNetFormer

This Keras code is for the paper A. Jamali, S. K. Roy, J. Li and P. Ghamisi, "[Neighborhood Attention Makes the Encoder of ResUNet Stronger for Accurate Road Extraction]," in IEEE Geoscience and Remote Sensing Letters, doi: 10.1109/LGRS.2024.3354560 [https://ieeexplore.ieee.org/document/10400502].
Apache License 2.0
13 stars 1 forks source link

Cannot Reproduce Correctly #5

Open WaOnEmperoR opened 4 months ago

WaOnEmperoR commented 4 months ago

Hi, I've been tried to reproduce your work but unfortunately I didn't get the same result as the paper despite using the same notebook and dataset from kaggle, which is the same as https://www.cs.toronto.edu/. Below is the result of my training process.  

Epoch 11/20 720/720 [==============================] - 82s 113ms/step - loss: 0.1940 - accuracy: 0.9156 - precision: 0.0000e+00 - recall: 0.0000e+00 - f1: 2.8445e-07 - val_loss: 0.1856 - val_accuracy: 0.9201 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - val_f1: 0.0000e+00 Epoch 12/20 720/720 [==============================] - 82s 114ms/step - loss: 0.1934 - accuracy: 0.9156 - precision: 0.1077 - recall: 7.8153e-07 - f1: 1.2056e-05 - val_loss: 0.1861 - val_accuracy: 0.9201 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - val_f1: 0.0000e+00 Epoch 13/20 720/720 [==============================] - 82s 113ms/step - loss: 0.1938 - accuracy: 0.9156 - precision: 0.0000e+00 - recall: 0.0000e+00 - f1: 2.9181e-06 - val_loss: 0.1850 - val_accuracy: 0.9201 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - val_f1: 0.0000e+00 Epoch 14/20 720/720 [==============================] - 82s 114ms/step - loss: 0.1937 - accuracy: 0.9156 - precision: 0.0000e+00 - recall: 0.0000e+00 - f1: 2.5722e-06 - val_loss: 0.1854 - val_accuracy: 0.9201 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - val_f1: 0.0000e+00 Epoch 15/20 720/720 [==============================] - 82s 114ms/step - loss: 0.1934 - accuracy: 0.9156 - precision: 0.0000e+00 - recall: 0.0000e+00 - f1: 4.9187e-06 - val_loss: 0.1851 - val_accuracy: 0.9201 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - val_f1: 0.0000e+00 Epoch 16/20 720/720 [==============================] - 82s 113ms/step - loss: 0.1935 - accuracy: 0.9156 - precision: 0.0000e+00 - recall: 0.0000e+00 - f1: 3.5134e-06 - val_loss: 0.1857 - val_accuracy: 0.9201 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - val_f1: 6.9142e-04 Epoch 17/20 720/720 [==============================] - 82s 114ms/step - loss: 0.1935 - accuracy: 0.9156 - precision: 0.0000e+00 - recall: 0.0000e+00 - f1: 2.4632e-06 - val_loss: 0.1849 - val_accuracy: 0.9201 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - val_f1: 0.0000e+00 Epoch 18/20 720/720 [==============================] - 82s 113ms/step - loss: 0.1933 - accuracy: 0.9156 - precision: 0.0000e+00 - recall: 0.0000e+00 - f1: 0.0000e+00 - val_loss: 0.1862 - val_accuracy: 0.9201 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - val_f1: 0.0000e+00 Epoch 19/20 720/720 [==============================] - 82s 114ms/step - loss: 0.1933 - accuracy: 0.9156 - precision: 0.0000e+00 - recall: 0.0000e+00 - f1: 0.0000e+00 - val_loss: 0.1855 - val_accuracy: 0.9201 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - val_f1: 0.0000e+00 Epoch 20/20 720/720 [==============================] - 82s 113ms/step - loss: 0.1932 - accuracy: 0.9156 - precision: 0.0000e+00 - recall: 0.0000e+00 - f1: 0.0000e+00 - val_loss: 0.1861 - val_accuracy: 0.9199 - val_precision: 0.1294 - val_recall: 2.2269e-04 - val_f1: 0.0012

aj1365 commented 4 months ago

Hi,

  1. You need to use this dataset (https://www.kaggle.com/datasets/balraj98/massachusetts-roads-dataset).
  2. I have modified the dataset as described in the paper. 3. Try to use more epochs. 4. I will check if I have the weights to be shared.