[cls] or pooled embedding?

[EdwardChan5000]

When i using event_cls to access a classification, the predicted labels come from cls or pooled embedding? If you can point that part of code to me, would be great helpful to me. Thanks.

[JayYip]

In most cases, [CLS] token embedding and pooled embedding are the same.

But some transformer model outputs don't have pooler outputs. In that case, pooler outputs would be token-wise average embedding.

https://github.com/JayYip/m3tl/blob/master/m3tl/modeling.py#L54 https://github.com/JayYip/m3tl/blob/master/m3tl/problem_types/cls.py#L38