search

jankrepl / mildlyoverfitted

Paper implementations from scratch and machine learning tutorials
MIT License
340 stars 124 forks source link

Mismatch of number of parameters #18

Closed kabbas570 closed 10 months ago

kabbas570 commented 2 years ago

Hey, the number of parameters in your custom model are not matching with nay of the model here?

https://pytorch.org/vision/stable/models/vision_transformer.html

jankrepl commented 1 year ago

Hey, the number of parameters in your custom model are not matching with nay of the model here?

https://pytorch.org/vision/stable/models/vision_transformer.html

I have never used torchvision ViT models before so cannot really tell you why that is the case without without inspecting things in detail. The video is based on timm.

Anyway, I suggest you go module by module and find the first one where the # parameters disagree. It will most likely be related to some different hyperparameter

vgoklani commented 1 year ago

Hey there @jankrepl (yes I'm trolling) I miss your YouTube videos! Hope you are planning to do another one soon! I hear there's this new thing called Stable Diffusion...

jankrepl commented 1 year ago

Hey there @jankrepl (yes I'm trolling) I miss your YouTube videos! Hope you are planning to do another one soon! I hear there's this new thing called Stable Diffusion...

@vgoklani Hehe:) Thanks, I got a bit lazy recently. I will try to go back to more frequent uploads:)