Closed mcshen99 closed 5 years ago
Which model is this?
I was following the example https://github.com/pytorch/fairseq/tree/bi_trans_lm/examples/pretraining for abstractive summarization with my own dataset.
@edunov ?
are you on the bi_trans_lm branch?
yes, i am
fixed, thanks for reporting
are you on the bi_trans_lm branch?
How can I on the bi_trans_lm branch? There is an error: invalid choice: 'bi_transformer_lm_big'. Thanks!
Hi, I am currently trying to use the pretrained cnn_dailynews language model. However, when I ran the command for training a final seq2seq model:
python train.py /private/home/edunov/cnn-dailymail/cnn-dailymail/finished_files/processed_nc_cnn --fp16 \ --no-enc-token-positional-embeddings --elmo-affine --share-decoder-input-output-embed \ --distributed-world-size 32 --distributed-port 17453 --no-progress-bar --max-update 30000 \ --optimizer adam --adam-betas '(0.9, 0.98)' --skip-invalid-size-inputs-valid-test \ --lr-scheduler inverse_sqrt --warmup-init-lr 1e-07 --warmup-updates 4000 --lr 0.0005 \ --ddp-backend no_c10d --min-lr 1e-09 --clip-norm 0.0 --dropout 0.3 --weight-decay 0.0 \ --criterion label_smoothed_cross_entropy --label-smoothing 0.1 --update-freq 4 --attention-dropout 0.2 \ --elmo-dropout 0.2 --max-tokens 3584 --arch transformer_wmt_en_de --seed 1 --warmup-init-lr 1e-7 \ --encoder-embed-path elmo:${LM_CHECKPOINT_PATH} --source-lang source --target-lang target
I receive an error that the elmo-dropout is not recognized as an argument. I was wondering if some code hadn't been pushed yet or if there might be another issue.