`Training Loss` and `Validation Loss` are not logged

[gxcuit]

Hi,

I'm following the quick start guide. And I add the logging_steps and eval_steps as the TrainingArguments. However, the Training Loss and Validation Loss are not logged.

Details

``` from setfit import SetFitModel model = SetFitModel.from_pretrained("BAAI/bge-small-en-v1.5", cache_dir="/kaggle/working/") from datasets import load_dataset dataset = load_dataset("SetFit/sst2") from setfit import sample_dataset train_dataset = sample_dataset(dataset["train"], label_column="label", num_samples=8) train_dataset # test_dataset = dataset["test"] test_dataset = dataset["test"].train_test_split(test_size=0.01)["test"] model.labels = ["negative", "positive"] test_dataset ``` ``` from setfit import TrainingArguments args = TrainingArguments( batch_size=32, num_epochs=100, eval_steps=10, logging_steps=10 ) from setfit import Trainer trainer = Trainer( model=model, args=args, train_dataset=train_dataset, eval_dataset=test_dataset, ) ```

[OrlerEmanuel]

Hello, I'm having the same problem, I'm working on Kaggle and bge-small-en-v1.5 as well. Did you find a solution for this?

[pm2111]

Same issue on my end!

[OrlerEmanuel]

Hey! If you run it on the terminal, you can see the eval_loss. Not sure if this is a bug or just a problem with my Kaggle environment. I also tried getting the log through a callback, but no luck.

[tomaarsen]

Hmm, I'm getting this on Google Colab as well. That's frustrating. I think it's because SetFit uses e.g. embedding_loss and eval_embedding_loss rather than loss and eval_loss.

• Tom Aarsen