When I finetune the bart large on cnn data, the generated samples looks like incomplete sentences

[songwang41]

❓ Questions and Help

I followed the BART_summarization.md, https://github.com/pytorch/fairseq/blob/master/examples/bart/README.summarization.md. However, when I applied the finedtune to make predictions, the summaries are not incomplete sentences. I have redo this twice and I observe this patterns (this happens even after many iterations, this once is just 2nd epochs). Any went wrong?

-for-be inmates are being held in Florida, Florida, prison, jail . They are being used in prison for mental illness, mental illness and mental illness . Judge says there are no charges of mental illness in prison . Leifman: "I don't know what we're going to do what you're going"
 of the film "Harry Potter" says he won't be able to buy the film . The 18-year-old says he has been able to sell for more than £20 million . The actor says he's not able to get the money on the film, but he says he will be able .
 of the bridge fell from the bridge . "It was a way to get off," says a woman . "I'm going to get to the ground," he says . "He was going to see me," says one of the driver, who was injured . The driver was injured in the crash, a woman says .
: "I don't know what I don't want to do," his mother says . Yifif, 5, was arrested Friday in Iraq for severe burns to his family . His mother says his family is "no reason" for his family to help him . "We want to go to the U.S. to the United States," his father says .
 was sent to the National Medical Center on Saturday morning . Doctors are expected to resume the procedure on Saturday . The doctor is expected to be removed from the White House on June 9 . The procedure is expected at 9.21 p.m. Saturday . It's unclear whether the procedure will be used for cancer .
 deal will be set to appear in court on Friday, court says . Judge: "It's not clear what he will have to do with your actions" A judge says he will not be charged in the case . The NFL says he has been charged with killing two dogs, including two dogs .

Here are ground truths from cnn_dm data.

Mentally ill inmates in Miami are housed on the "forgotten floor" Judge Steven Leifman says most are there as a result of "avoidable felonies" While CNN tours facility, patient shouts: "I am the son of the president" Leifman says the system is unjust and he's fighting for change .
Harry Potter star Daniel Radcliffe gets £20M fortune as he turns 18 Monday . Young actor says he has no plans to fritter his cash away . Radcliffe's earnings from first five Potter films have been held in trust fund .
NEW: "I thought I was going to die," driver says . Man says pickup truck was folded in half; he just has cut on face . Driver: "I probably had a 30-, 35-foot free fall" Minnesota bridge collapsed during rush hour Wednesday .
Parents beam with pride, can't stop from smiling from outpouring of support . Mom: "I was so happy I didn't know what to do" Burn center in U.S. has offered to provide treatment for reconstructive surgeries . Dad says, "Anything for Youssif"
Five small polyps found during procedure; "none worrisome," spokesman says . President reclaims powers transferred to vice president . Bush undergoes routine colonoscopy at Camp David .
NEW: NFL chief, Atlanta Falcons owner critical of Michael Vick's conduct . NFL suspends Falcons quarterback indefinitely without pay . Vick admits funding dogfighting operation but says he did not gamble . Vick due in federal court Monday; future in NFL remains uncertain .
Aid workers: Violence, increased cost of living drive women to prostitution . Group is working to raise awareness of the problem with Iraq's political leaders . Two Iraqi mothers tell CNN they turned to prostitution to help feed their children . "Everything is for the children," one woman says .

My training command:

TOTAL_NUM_UPDATES=20000  
WARMUP_UPDATES=500      
LR=3e-05
MAX_TOKENS=2048
UPDATE_FREQ=8
BART_PATH=pretrained/bart.large/model.pt
DATA_DIR=binarized
SAVE_DIR=checkpoints

CUDA_VISIBLE_DEVICES=0,1,2,3 fairseq-train $DATA_DIR \
    --restore-file $BART_PATH \
    --save-dir $SAVE_DIR \
    --max-tokens $MAX_TOKENS \
    --task translation \
    --source-lang source --target-lang target \
    --truncate-source \
    --layernorm-embedding \
    --share-all-embeddings \
    --share-decoder-input-output-embed \
    --reset-optimizer --reset-dataloader --reset-meters \
    --required-batch-size-multiple 1 \
    --arch bart_large \
    --criterion label_smoothed_cross_entropy \
    --label-smoothing 0.1 \
    --dropout 0.1 --attention-dropout 0.1 \
    --weight-decay 0.01 --optimizer adam --adam-betas "(0.9, 0.999)" --adam-eps 1e-08 \
    --clip-norm 0.1 \
    --lr-scheduler polynomial_decay --lr $LR --total-num-update $TOTAL_NUM_UPDATES --warmup-updates $WARMUP_UPDATES \
    --fp16 --update-freq $UPDATE_FREQ \
    --skip-invalid-size-inputs-valid-test \
    --find-unused-parameters;

[stale[bot]]

This issue has been automatically marked as stale. If this issue is still affecting you, please leave any comment (for example, "bump"), and we'll keep it open. We are sorry that we haven't been able to prioritize it yet. If you have any new additional information, please include it with your comment!

[stale[bot]]

Closing this issue after a prolonged period of inactivity. If this issue is still present in the latest release, please create a new issue with up-to-date information. Thank you!