Open Jeffkang-94 opened 1 year ago
Hello @Jeffkang-94,
Sorry for my delay in the response, and thanks a lot for raising the issue !
It should be solved at : https://github.com/facebookresearch/vissl/commit/04788de934b39278326331f7a4396e03e85f6e55
I also fixed the configurations that had the wrong MLP_DIM!
Thank you, Quentin
Thank you for your efforts! I think we can wrap-up this issue then.
Instructions To Reproduce the Issue:
Dino
configuration contains the parameter namedMLP_DIM
which looks like being adjustable by the user. But actually, it is hard coded. See the line here https://github.com/facebookresearch/vissl/blob/4ca0ae351b1030742132179bffd9b8640fd5a6f0/vissl/models/trunks/vision_transformer.py#L149-L155I wanted to adjust the
mlp_ratio
or size ofmlp_dim
through the configuration as I've met several OOM issues because of the huge dataset,But it seems that the mlp_dim is not adjustable through the configuration. Apart from it, it also raises some confusion for the user who wants to change the size of mlp dimension.
Expected behavior:
To be specific, with the configuration
mlp
is supposed to havebut actually it has
because of the pre-defined value
mlp_ratio=4.0
here https://github.com/facebookresearch/vissl/blob/4ca0ae351b1030742132179bffd9b8640fd5a6f0/vissl/models/trunks/vision_transformer.py#L262