Possibility to monitor head training?

huggingface / setfit

Efficient few-shot learning with Sentence Transformers

https://hf.co/docs/setfit

Apache License 2.0

2.17k stars 220 forks source link

Possibility to monitor head training? #525

Open Ulipenitz opened 4 months ago

Ulipenitz commented 4 months ago

Hello! Is there a plan to implement monitoring of the head training phase as well? Currently it seems that logging (Wandb, neptune etc.) stops after the body training, but it can be useful to see the logs & implement early stopping for the head as well. Especially for end-to-end training when you use more complex pytorch heads where you modify the architecture during hp-search.

kgourgou commented 3 months ago

Hello.

Are you using the default head? It may just be that this happens because the default head is an sklearn head (last I checked).

Ulipenitz commented 3 months ago

Hello! No, I am using a custom pytorch head.