Your config.json file will cause the model to error out on contexts larger than 2048 tokens. - Githubissues

m42-health / med42

MIT License

40 stars 0 forks source link

Your config.json file will cause the model to error out on contexts larger than 2048 tokens. #3

Open scottfleming opened 8 months ago

scottfleming commented 8 months ago

The offending line is here: https://huggingface.co/m42-health/med42-70b/blob/ce666e125280bad558a81938378252fc1d602c77/config.json#L12

See https://huggingface.co/meta-llama/Llama-2-70b-chat-hf/commit/c733a855493ee517f3c51c490ea6ce6ce52cd837 and https://github.com/huggingface/transformers/issues/24986