Open mdingemanse opened 9 months ago
Now has an answer:
The instruction-tuned model was trained on instructions from the P3 and Natural Instructions datasets, which were decontaminated against HELM (you can find more details about the decontamination strategy in this blog bost).
You can find the resulting decontaminated dataset used to train the instruction tuned model here: https://huggingface.co/datasets/togethercomputer/RedPajama-Data-Instruct -- the metadata here contains a source field pointing to the task / dataset where the instance comes from.
Erasmian Language Model (ELM): https://github.com/Joaoffg/ELM/issues/1 (and see #65)
Nanbeige Chat: https://huggingface.co/Nanbeige/Nanbeige2-8B-Chat/discussions/2#6621e15a4d17641cf788cbd5
Update: request for training data was closed without comment. https://huggingface.co/Nanbeige/Nanbeige2-8B-Chat/discussions/1
Update: request for post-training data was answered and closed
This issue is just a place to keep track of issues filed at other github projects or HuggingFace hubs to ask for documentation