m42-health / med42

MIT License
40 stars 0 forks source link

How many tokens was this model trained on? #2

Open abhi-mosaic opened 10 months ago

abhi-mosaic commented 10 months ago

Hi Med42 team, congrats on building this model, it looks fantastic and thank you for sharing with the community!

To help guide developers who may want their own domain-specific models, could you help provide some details on how many tokens Med42 was finetuned on? In the HF Model card (https://huggingface.co/m42-health/med42-70b) I saw a note that the dataset contains 250M tokens, but was the model trained for just 1 epoch or multiple epochs?

Thank you for your help!