Dependent on @rbiswasfc MLMMLU branch being merged.
Minor scripts to automatically convert train configs from checkpoints, as well as an example .sh to do so on a folder full of many checkpoints.
The script is built from @NohTow's convert script and handles a few more cases. The script takes a few arguments:
--checkpoint, the path to a checkpoint (either folder containing it or path to the .pt file within this folder directly)
--wandb_run (optional), the associated wandb run
--wandb_project (optional, defaults to "bert24") if the run isn't just in the shove-all bert24 (so you should use this one!)
--train_config (optional) the path to the train config used to train the checkpoint
--output_dir (optional, defaults to ".eval_configs/"), the path where the eval config will be output
Finding the train config goes through a priority procesS:
If provided, it'll use train_config
If not, it'll look for a config .yaml file with the same name as the checkpoint
If provided, it will load the config.yaml associated with the wandb_run name given (in the wandb_project given)
If none of the above matches, it'll try to find a wandb run named the same as the checkpoint name within the given wandb_project and load the config from there.
Tests
WIP because I haven't yet been able to test the script (some admin issue so I gotta run to the US consulate) and I am 99% sure there will be some minor issues, but overall should be fine with no major changes required.
Changes
Dependent on @rbiswasfc MLMMLU branch being merged.
Minor scripts to automatically convert train configs from checkpoints, as well as an example
.shto do so on a folder full of many checkpoints.The script is built from @NohTow's convert script and handles a few more cases. The script takes a few arguments:
--checkpoint, the path to a checkpoint (either folder containing it or path to the .pt file within this folder directly)--wandb_run(optional), the associated wandb run--wandb_project(optional, defaults to"bert24") if the run isn't just in the shove-allbert24(so you should use this one!)--train_config(optional) the path to the train config used to train the checkpoint--output_dir(optional, defaults to".eval_configs/"), the path where the eval config will be outputFinding the train config goes through a priority procesS:
train_config.yamlfile with the same name as the checkpointconfig.yamlassociated with thewandb_runname given (in thewandb_projectgiven)wandb_projectand load the config from there.Tests
WIP because I haven't yet been able to test the script (some admin issue so I gotta run to the US consulate) and I am 99% sure there will be some minor issues, but overall should be fine with no major changes required.