This PR by @Stillerman added support to FIM for santacoder fine-tuning:rocket:
But since this code is used not just for SantaCoder this PR changes default fim rates to zero, and handles case where user specifies fim_rate>0 and tokenizer doesn't have fim tokens.
Stillerman
commented
1 year ago
This PR by @Stillerman added support to FIM for santacoder fine-tuning:rocket: But since this code is used not just for SantaCoder this PR changes default fim rates to zero, and handles case where user specifies
fim_rate>0and tokenizer doesn't have fim tokens.