classification mobilevit_v2 model，where can I find softmax activation layer？

[chenying99]

cvnets/models/classification/mobilevit_v2.py the last layer as follow:

https://github.com/apple/ml-cvnets/blob/main/cvnets/models/classification/mobilevit_v2.py#L90

self.classifier = nn.Sequential( GlobalPool(pool_type=pool_type, keep_dim=False), LinearLayer(in_features=out_channels, out_features=num_classes, bias=True), )

where can I find softmax activation layer？ thank you

[farzadab]

Since computing the loss after SoftMax is numerically unstable, usually the softmax is not found in the models themselves and this step usually happens implicitly in the loss function.

Take a look at the documentation of CrossEntropyLoss in PyTorch. It takes logits as input, not actual probabilities. You can compare that with NLLLoss which does expect probabilities.

Please note that although SoftMax is not added during training, it can be added in the inference/export step as needed.

[chenying99]

OK, thank you for your reply, I will refer to relevant documents

[farzadab]

Np. Closing the issue. Feel free to reopen if need be.