Closed Muedi closed 1 year ago
Just as @jamaliki pointed out I changed the tokens to be correct.
I left the whitespaces for now, but this can be removed deleting one line.
When the Tokenizer Pull request is though, I'll try to use this instead of the pretrained esm2 one?
best, Max
Just as @jamaliki pointed out I changed the tokens to be correct.
I left the whitespaces for now, but this can be removed deleting one line.
When the Tokenizer Pull request is though, I'll try to use this instead of the pretrained esm2 one?
best, Max