tanmay-ty / SpectralNET

49 stars 24 forks source link

Train-test-set split #2

Closed leonvarga closed 2 years ago

leonvarga commented 2 years ago

Hello Tanmay, hello Utkarsh,

your work is an interesting contribution to the small research field of hyperspectral imaging, and I really like your approach.

I am currently working on a similar technique and would like to evaluate my method on the Hyperspectral Remote Sensing Scenes dataset as well. While reviewing your code, I came across your patch generation and train-val test split, which seems kind of strange to me.

You generate patches with a stride of 1, which results in a lot of overlap between patches. By randomly splitting these patches into train and test set, the training set contains patches that are very similar to the patches in the test set. Therefore, you cannot use these experiments to prove the generalization properties of your method.

Could you please clarify if I missed something here? Otherwise, the claim of "state of the art" is misleading.

Yours sincerely, Leon Varga

tanmay-ty commented 2 years ago

Hello Leon, I understand your concern, but if you look carefully we only use at most 30% of the patches generated for training. Thus even if there maybe some overlapping due to the random test train split (which was used for convenience) the model has not seen 60% of the data. I hope this answers your question about generalization. Thanks, Tanmay