Checkpoints for models trained on Spider

[awasthiabhijeet]

Dear Authors,

Thanks for putting together this great resource.

I wish to know if we have checkpoints released for T5 models finetuned on the Spider dataset. i.e. the checkpoints corresponding to the Spider row in Table 2.

The Spider checkpoints currently released on huggingface correspond only to prefix-tuning (and not finetuning) experiments. Am I right?

Also, what config files should one use to reproduce Spider row in Table 2?

E.g. is T5_large_finetune_spider_with_cell_value.cfg the file reproduce numbers for T5 large?

Thank you :)

[ChenWu98]

The following two sets of commands are for T5-base and T5-large. Please let us know if there are any further issues!

export RUN_NAME=T5_base_finetune_spider_with_cell_value
export SEED=2
python -m torch.distributed.launch --nproc_per_node 1 --master_port 1234 train.py --seed $SEED --cfg Salesforce/$RUN_NAME.cfg --run_name $RUN_NAME$SEED --logging_strategy steps --logging_first_step true --logging_steps 4 --evaluation_strategy steps --eval_steps 500 --metric_for_best_model avr --greater_is_better true --save_strategy steps --save_steps 500 --save_total_limit 1 --load_best_model_at_end --gradient_accumulation_steps 8 --num_train_epochs 200 --adafactor true --learning_rate 5e-5 --do_train --do_eval --do_predict --predict_with_generate --output_dir output/$RUN_NAME$SEED --overwrite_output_dir --per_device_train_batch_size 4 --per_device_eval_batch_size 2 --generation_num_beams 1 --generation_max_length 512 --input_max_length 512 --ddp_find_unused_parameters true

export RUN_NAME=T5_large_finetune_spider_with_cell_value
export SEED=2
python -m torch.distributed.launch --nproc_per_node 1 --master_port 1250 train.py --seed $SEED --cfg Salesforce/$RUN_NAME.cfg --run_name $RUN_NAME$SEED --logging_strategy steps --logging_first_step true --logging_steps 4 --evaluation_strategy steps --eval_steps 500 --metric_for_best_model avr --greater_is_better true --save_strategy steps --save_steps 500 --save_total_limit 1 --load_best_model_at_end --gradient_accumulation_steps 16 --num_train_epochs 200 --adafactor true --learning_rate 5e-5 --do_train --do_eval --do_predict --predict_with_generate --output_dir output/$RUN_NAME$SEED --overwrite_output_dir --per_device_train_batch_size 2 --per_device_eval_batch_size 2 --generation_num_beams 1 --generation_max_length 512 --input_max_length 512 --ddp_find_unused_parameters true

[awasthiabhijeet]

Thanks @ChenWu98 ,

Would it be possible to share the pre-trained ckpts for these models as well? Finetuning T5-large on Spider for longer epochs would be somewhat difficult for us.

Thanks again! :)

[ChenWu98]

I edited the two sets of commands above since I forgot to add `_with_cell_value``. Sorry for the confusion.

Sadly, we ran these experiments on cloud servers and didn't anticipate the value of saving checkpoints then. For Spider, we recommend https://github.com/ElementAI/picard if you are interested.

Good luck!