search

ContextualAI / gritlm

Generative Representational Instruction Tuning
https://arxiv.org/abs/2402.09906
MIT License
479 stars 33 forks source link

BUG: NotImplementedError #8

Open oYoungCo opened 4 months ago

oYoungCo commented 4 months ago

When I run the script of Training.embedding_model, a bug is occurred. How can I fix it? File /gritlm/training/run.py", line 166, in main else: raise NotImplementedError NotImplementedError

Muennighoff commented 4 months ago

if you check line 166 you see that it's cuz your training data is not in the right format, what data are you using & whats ur script?

oYoungCo commented 4 months ago

if you check line 166 you see that it's cuz your training data is not in the right format, what data are you using & whats ur script? I run this script, and data is toy_data provided in code. torchrun --nproc_per_node 1 \ -m training.run \ --output_dir test_path \ --model_name_or_path openaccess-ai-collective/tiny-mistral \ --train_data training/toy_data \ --learning_rate 1e-5 \ --num_train_epochs 5 \ --per_device_train_batch_size 2 \ --dataloader_drop_last True \ --normalized True \ --temperature 0.02 \ --query_max_len 32 \ --passage_max_len 128 \ --train_group_size 2 \ --negatives_cross_device \ --mode unified

Muennighoff commented 4 months ago

It should have been fixed now (by just removing the raise)! Thanks for raising this!