AttributeError: 'SetFitHead' object has no attribute 'multitarget'

[Aryavir07]

# Load a SetFit model from Hub
model = SetFitModel.from_pretrained(
    "/content/drive/MyDrive/custom_model",
    use_differentiable_head=True,
    head_params={"out_features": 2},
)
model.sequence_length = 128

trainer = SetFitTrainer(
    model=model,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    loss_class=losses.MultipleNegativesRankingLoss(model),
    metric="accuracy",
    batch_size=8,
    num_iterations=20, # The number of text pairs to generate for contrastive learning
    num_epochs=10, # The number of epochs to use for contrastive learning
    # column_mapping={"text": "text", "labels": "label"} # Map dataset columns to text/label expected by trainer
)

trainer.unfreeze(keep_body_frozen=False)

trainer.train(
    num_epochs=5, # The number of epochs to train the head or the whole model (body and head)
    batch_size=64,
    max_length = 128,
    body_learning_rate=1e-5, # The body's learning rate
    learning_rate=1e-4, # The head's learning rate
    l2_weight=0.0, # Weight decay on **both** the body and head. If `None`, will use 0.01.
)

Outputs: "AttributeError: 'SetFitHead' object has no attribute 'multitarget'"

Above, "custom_model" is the "sentence-transformers/all-mpnet-base-v2" trained on the same hyperparameters and SetFit release version v0.5.0.

The problem was resolved once I installed SetFit v0.5.0, but it persists in SetFit v0.6.0.

[tomaarsen]

Did you resolve this issue? No need to close this if not. I assume others may encounter this issue, too.

[Aryavir07]

Did you resolve this issue? No need to close this if not. I assume others may encounter this issue, too. Reopening it.

[tomaarsen]

The new v0.6.0 does introduce this new value. It's a bit unfortunate that this may mean that SetFit v0.5.0 can't easily be loaded on v0.6.0 onwards, especially as we introduce further changes from v0.5.0. I'm not quite sure how best to tackle this.

[Aryavir07]

Therefore, one should only load the model on version 0.5.0 if one's model was trained on version 0.5.0; otherwise, it will encounter this error.

This also applies to the model's conversion to ONNX format, where the SetFit version difference results in the error "AttributeError: 'SetFitHead' object has no attribute 'eps' ".

@tomaarsen