search

pytorch / examples

A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.
https://pytorch.org/examples
BSD 3-Clause "New" or "Revised" License
22k stars 9.49k forks source link

vision-transformer problem report #1184

Open ChenDaiwei-99 opened 10 months ago

ChenDaiwei-99 commented 10 months ago

image image image

msaroufim commented 10 months ago

If you wanna contribute some fixes would be happy to merge

ChenDaiwei-99 commented 10 months ago

Sure, will do it in a few days :)

colinosterman commented 8 months ago

I can confirm this example is badly broken. I added some code to compare individual labels to predictions and discovered the forward pass of ViT always returns the same tensor. No matter the input. The tensor it returns is different each time I run it, even if I load the same weights from the save file and don't do any training. It's no wonder it can't do better than 2.3. Always giving the same prediction should accidentally hit on the correct label about as often as random guessing.

fanqieguo commented 1 month ago

I added the printout of accuracy in the code, but during the training process, the accuracy does not improve, and the loss does not converge, even after training for many epochs. Does this model really work? I think this model has serious problems. I hope to get an answer, this is very important to me