search

ZiJ-Wang / VQSRS

The official code for "Fast Real‑Time Brain Tumor Detection Based on Stimulated Raman Histology and Self‑Supervised Deep Learning Model".
3 stars 0 forks source link

Accuracy Drops with shuffle=False After Training for Heatmap Generation #3

Open MshGhazale opened 1 month ago

MshGhazale commented 1 month ago

Hello,

After training my model, I needed to generate a heatmap. To ensure the correct order of the data for plotting the heatmap, I set shuffle=False during evaluation. However, I noticed a significant drop in the model’s accuracy on the test data after doing this.

Could you please explain the proper process for generating a heatmap without affecting the model’s performance, or provide us with the correct code for generating the heatmap?

Additionally, I would like to understand why shuffling the data impacts the model’s accuracy. What is the reason for this accuracy drop when shuffle=False, and how can this be properly handled?

Your guidance would be very helpful. Thank you!

ZiJ-Wang commented 4 days ago

Hey, I’m really sorry, I just saw your message！

When setting shuffle=False during evaluation, you should add model.eval() after model = model.to(device). If you don't, certain operations (like Dropout and BatchNorm) that are specific to training mode will be enabled, which can affect the accuracy when shuffle=False.

Once I have everything organized, I'll upload the heatmap code. Thank you for your patience!

ZiJ-Wang commented 3 days ago

Hya,

We’ve uploaded the code for generating the heatmap, so feel free to give it a try! It should help with the process.