search

arcee-ai / mergekit

Tools for merging pretrained large language models.
GNU Lesser General Public License v3.0
4.88k stars 446 forks source link

RuntimeError: Unsupported architecture BertForSequenceClassification #381

Open lrsbrgrn opened 4 months ago

lrsbrgrn commented 4 months ago

Hi,

Trying to merge multiple BERT models in scope of text-classification.

When doing a simple linear merge, i got the following error message RuntimeError: Unsupported architecture BertForSequenceClassification.

Having a closer look in the lib, it seems this architecture should be supported by design as you can see in bert-sequence-classification.json

Any insights?

Many thanks,

LB