Open allanj opened 2 years ago
I run the script with the following command under the graph2tree folder:
graph2tree
python3 -m src.main -mode train -gpu 0 -embedding roberta -emb_name roberta-base -hidden_size 384 -depth 2 -lr 8e-4 -emb_lr 1e-5 -batch_size 8 -epochs 50 -dataset mawps-asdiv-a_svamp -no-full_cv -run_name run_svamp
During training, it shows the extra numbers are:
['0.01', '12.0', '1.0', '100.0', '0.1', '0.5', '3.0', '4.0', '7.0']
['1.0', '0.1', '3.0', '5.0', '0.5', '12.0', '4.0', '60.0', '25.0', '0.01', '0.05', '2.0', '10.0', '0.25', '8.0', '7.0', '100.0']
Looks a bit inconsistent.
<Trial 3754768 worker_0> graph2tree $ tail -n 100 svamp.log nohup: ignoring input 2021-11-10 02:27:24,969 | INFO | main.py: 420 : main() :: Experiment Name: run_svamp 2021-11-10 02:27:24,969 | DEBUG | main.py: 421 : main() :: Created Relevant Directories 2021-11-10 02:27:24,969 | INFO | main.py: 423 : main() :: Loading Data... 2021-11-10 02:27:26,430 | DEBUG | main.py: 429 : main() :: Data Loaded... 2021-11-10 02:27:26,430 | DEBUG | main.py: 431 : main() :: Number of Training Examples: 3138 2021-11-10 02:27:26,430 | DEBUG | main.py: 432 : main() :: Number of Testing Examples: 1000 2021-11-10 02:27:26,430 | DEBUG | main.py: 433 : main() :: Extra Numbers: ['0.01', '12.0', '1.0', '100.0', '0.1', '0.5', '3.0', '4.0', '7.0'] 2021-11-10 02:27:26,430 | DEBUG | main.py: 434 : main() :: Maximum Number of Numbers: 7 2021-11-10 02:27:26,430 | INFO | main.py: 437 : main() :: Creating Vocab... 2021-11-10 02:27:27,462 | DEBUG | pre_data.py: 611 : prepare_data() :: Indexed 4071 words in input language, 21 words in output 2021-11-10 02:27:27,515 | DEBUG | helper.py: 169 : get_latest_checkpoint() :: Checkpoint found at : models/run_svamp/model.pt 2021-11-10 02:27:27,516 | DEBUG | main.py: 460 : main() :: Vocab saved at models/run_svamp/vocab1.p 2021-11-10 02:27:27,517 | DEBUG | main.py: 472 : main() :: Config File Saved 2021-11-10 02:27:27,517 | INFO | main.py: 474 : main() :: Initializing Models... Some weights of the model checkpoint at roberta-base were not used when initializing RobertaModel: ['lm_head.layer_norm.bias', 'lm_head.decoder.weight', 'lm_head.dense.bias', 'lm_head.dense.weight', 'lm_head.layer_norm.weight', 'lm_head.bias'] - This IS expected if you are initializing RobertaModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model). - This IS NOT expected if you are initializing RobertaModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). 2021-11-10 02:27:37,520 | DEBUG | main.py: 491 : main() :: Models Initialized 2021-11-10 02:27:37,521 | INFO | main.py: 492 : main() :: Initializing Optimizers... 2021-11-10 02:27:37,522 | DEBUG | main.py: 500 : main() :: Optimizers Initialized 2021-11-10 02:27:37,522 | INFO | main.py: 501 : main() :: Initializing Schedulers... 2021-11-10 02:27:37,522 | DEBUG | main.py: 509 : main() :: Schedulers Initialized 2021-11-10 02:27:37,522 | INFO | main.py: 511 : main() :: Loading Models on GPU 0... 2021-11-10 02:27:45,194 | DEBUG | main.py: 521 : main() :: Models loaded on GPU 0 2021-11-10 02:27:45,195 | INFO | main.py: 529 : main() :: Starting Training Procedure 2021-11-10 02:27:48,878 | INFO | logger.py: 33 : print_log() :: Epoch: 1 Transfer numbers... keep_words 4068 / 4068 = 1.0000 2021-11-10 02:29:46,859 | DEBUG | main.py: 555 : main() :: Training for epoch 0 completed... Time Taken: 0h 1m 57s 2021-11-10 02:29:46,860 | INFO | main.py: 582 : main() :: Starting Validation 2021-11-10 02:34:45,014 | INFO | helper.py: 86 : save_checkpoint() :: Saving Checkpoint at : models/run_svamp/model.pt 2021-11-10 02:34:49,094 | INFO | logger.py: 33 : print_log() :: Epoch: 1 best epoch: 1 train loss epoch: 1.0544757959705424 min train loss: 1.0544757959705424 train acc epoch: 0.0 max train acc: 0.0 val acc epoch: 0.267 equation acc epoch: 0.255 max val acc: 0.267 equation acc: 0.255 2021-11-10 02:34:49,094 | DEBUG | main.py: 691 : main() :: Validation Completed... Time Taken: 0h 5m 2s 2021-11-10 02:34:52,701 | INFO | logger.py: 33 : print_log() :: Epoch: 2 Completed 223 / 393...
I run the script with the following command under the
graph2treefolder:During training, it shows the extra numbers are:
Looks a bit inconsistent.