search

wangzhecheng / DeepSolar

Nationwide houseshold-level solar panel identification with deep learning
MIT License
247 stars 73 forks source link

Low Recall Rate on the test set. #15

Open billlyzhaoyh opened 4 years ago

billlyzhaoyh commented 4 years ago

I am running the model following the instruction on the test set and a personal dataset and find out that the FN rate is very high and while the precision rate is close to what is being reported in the paper, the recall rate is not nearly the same (10% and 40% for residential and commercial areas respectively).

Anyone know what the issue is?

fms-santos commented 4 years ago

Hi Billy, have you solved your low recall rate problem? I'm facing the same issue in a personal dataset in Portugal with a 28% recall for a residential area.

billlyzhaoyh commented 4 years ago

Haven't really solved this issue... it is the problem with not reproducible research...

fms-santos commented 4 years ago

@billlyzhaoyh Thanks for the feedback. I do agree with you. We've tried in another PT area (https://goo.gl/maps/buQxMkEgC2hNDDAKA) with around 10.5k tiles and we got a Precision of 90% and Recall of 36%. Perhaps we really have to train the model with our own images...