While running bradley-terry-rm/llama3_rm.py the final saved model does not have a lm head as the script is using a AutoModelForSequenceClassification model and not CausalLM. Because of this the lm head is initialized from scratch while loading the saved model. Is this correct or do you manually add the lm head weights?
Hello and thanks for your work!
While running bradley-terry-rm/llama3_rm.py the final saved model does not have a lm head as the script is using a AutoModelForSequenceClassification model and not CausalLM. Because of this the lm head is initialized from scratch while loading the saved model. Is this correct or do you manually add the lm head weights?