TextCatCNN.v2 doesn't work with transformers

[polm]

How to reproduce the behaviour

As brought up in #11925, if you use TextCatCNN.v2 with transformers you get an error like this:

ValueError: Cannot get dimension 'nI' for model 'linear': value unset

The issue is that when initializing the textcat, the linear layer in the model is resized once for each label added. When resized, the linear layer is detected as initialized, and is then re-initialized. However it's not actually initialized at that point, and is missing dimension information because of how transformers initialization works, so initialization fails.

For the time being, the workaround is to use TextCatCNN v1 or another architecture. The main difference in v2 is that it's resizable, so if you aren't using that particular feature performance shouldn't differ significantly.

Info about spaCy

• spaCy version: 3.4.3
• Platform: Linux-6.0.10-arch2-1-x86_64-with-glibc2.36
• Python version: 3.10.8
• Pipelines: en_coreference_web_trf (3.4.0a2), fi_core_news_sm (3.4.0), en_core_web_md (3.4.0), ca_core_news_sm (3.4.0), es_core_news_sm (3.4.0), ja_core_news_sm (3.4.0), fr_core_news_sm (3.4.0), en_core_web_trf (3.4.0), nl_core_news_sm (3.4.0), nb_core_news_sm (3.4.0), en_vectors_floret_md (0.0.1), de_core_news_sm (3.4.0), en_core_web_lg (3.4.1), en_core_web_sm (3.4.1), ko_core_news_sm (3.4.0), ru_core_news_sm (3.4.0)

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.