search

tianyi-lab / Cherry_LLM

[NAACL'24] Self-data filtering of LLM instruction-tuning data using a novel perplexity-based difficulty score, without using any other models
306 stars 21 forks source link

how many epochs to train on cherry data? #19

Closed menghonghan closed 10 months ago

menghonghan commented 10 months ago

I saw from the paper for pre-exprience model, your trained 1 epoch. But after data selection, it's fuzzy in the paper how many epochs you trained on cherry data, it that 3 ? Looking forward for your quick response.

MingLiiii commented 10 months ago

Thank you very much for your interest! Yes, we trained the cherry model for 3 epochs. You can find the detailed parameters in the Implementation Details section of our paper or the Hyperparameters section in our repo~

menghonghan commented 10 months ago

Thx

On Wed, Jan 3, 2024 at 12:26 Ming Li @.***> wrote:

Thank you very much for your interest! Yes, we trained the cherry model for 3 epochs. You can find the detailed parameters in the Implementation Details section of our paper or the Hyperparameters section in our repo~

— Reply to this email directly, view it on GitHub [github.com] https://urldefense.com/v3/__https://github.com/MingLiiii/Cherry_LLM/issues/19*issuecomment-1874833407__;Iw!!DaRZpAeNFA!eyXcFHWIZA7DtZEc4Ng24zL-B65BffrtZ4q_Qz-b19j2YTeBXejVLOk0ORP_b48kVKEruuqltCFlfxD3iVNCUPmXhLD2Eg$, or unsubscribe [github.com] https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AOUGLLBGRZYVBWL4ZQ5QTALYMTMXJAVCNFSM6AAAAABBK36YI6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZUHAZTGNBQG4__;!!DaRZpAeNFA!eyXcFHWIZA7DtZEc4Ng24zL-B65BffrtZ4q_Qz-b19j2YTeBXejVLOk0ORP_b48kVKEruuqltCFlfxD3iVNCUPnlzOPk_A$ . You are receiving this because you authored the thread.Message ID: @.***>