likyoo / change_detection.pytorch

Deep learning models for change detection of remote sensing images
MIT License
331 stars 59 forks source link

Evaluation with different thresholds give the same results #23

Open mikel-brostrom opened 2 years ago

mikel-brostrom commented 2 years ago

This piece of code :

for x in np.arange(0.6, 0.9, 0.1):
    print('Eval with TH:', x)
    metrics = [
        cdp.utils.metrics.Fscore(activation='argmax2d', threshold=x),
        cdp.utils.metrics.Precision(activation='argmax2d', threshold=x),
        cdp.utils.metrics.Recall(activation='argmax2d', threshold=x),
    ]

    valid_epoch = cdp.utils.train.ValidEpoch(
        model,
        loss=loss,
        metrics=metrics,
        device=DEVICE,
        verbose=True,
    )

    valid_logs = valid_epoch.run(valid_loader)
    print(valid_logs)

Give me the following result:

Eval with TH: 0.6
valid: 100%|█████████████████████████████████████████████████████████████| 505/505 [01:12<00:00,  6.98it/s, cross_entropy_loss - 0.08708, fscore - 0.8799, precision - 0.8946, recall - 0.8789]
{'cross_entropy_loss': 0.0870812193864016, 'fscore': 0.8798528309538921, 'precision': 0.8946225793644936, 'recall': 0.8789094516579565}

Eval with TH: 0.7
valid: 100%|█████████████████████████████████████████████████████████████| 505/505 [01:12<00:00,  6.99it/s, cross_entropy_loss - 0.08708, fscore - 0.8799, precision - 0.8946, recall - 0.8789]
{'cross_entropy_loss': 0.08708121913835626, 'fscore': 0.8798528309538921, 'precision': 0.8946225793644936, 'recall': 0.8789094516579565}

Eval with TH: 0.7999999999999999
valid: 100%|█████████████████████████████████████████████████████████████| 505/505 [01:11<00:00,  7.02it/s, cross_entropy_loss - 0.08708, fscore - 0.8799, precision - 0.8946, recall - 0.8789]
{'cross_entropy_loss': 0.08708121978843793, 'fscore': 0.8798528309538921, 'precision': 0.8946225793644936, 'recall': 0.8789094516579565}