search

abdulhaim / LMRL-Gym

MIT License
75 stars 9 forks source link

Chess train_full_games_bc script #14

Closed PioneerAlexander closed 8 months ago

PioneerAlexander commented 9 months ago

Hello,

For the training bc with chess full games I use the command line

python -m llm_rl_scripts.chess.bc.train_full_games_bc HF gpt2 dataset_path

Where the dataset is train_bc.jsonl

After running this command I get the similar issue to the other issue (Issue #7).The current version of script contains the IterableDataset instead of Dataset, which makes the training with the multiple epochs impossible. I should also note that the Issue #7 was resolved due to edited dataset creating. Could you please specify whether are you training the BC with IterableDataset or not, and if you train with the multiple epochs, how many are you using in your experiments?

I look forward to your response.

icwhite commented 9 months ago

Hi @PioneerAlexander when we run this we only use one epoch. We use IterableDataset as it speeds up the initialization of the dataset. However, if you want to use multiple epochs you should use Dataset.