模型初始化 - Githubissues

sunbo1999 commented 1 year ago

您好，请问如果我想用自己本地的模型初始化，而不是fnlp/bart-large-chinese，下面的参数应该做什么改变呢

CUDA_VISIBLE_DEVICES=0 nohup python -u $FAIRSEQ_CLI_PATH/train.py $PROCESSED_DIR_STAGE1/bin \
    --save-dir $MODEL_DIR_STAGE1 \
    --user-dir ../../src/src_syngec/syngec_model \
    --bart-model-file-from-transformers fnlp/bart-large-chinese \
    --task syntax-enhanced-translation \
    --arch syntax_enhanced_bart_large \
    --skip-invalid-size-inputs-valid-test \
    --max-tokens 2048 \
    --optimizer adam \
    --max-source-positions 512 \
    --max-target-positions 512 \
    --lr 3e-05 \
    --warmup-updates 2000 \
    -s src \
    -t tgt \
    --lr-scheduler polynomial_decay \
    --clip-norm 1.0 \
    --criterion label_smoothed_cross_entropy \
    --label-smoothing 0.1 \
    --max-epoch 60 \
    --share-all-embeddings \
    --adam-betas '(0.9,0.999)' \
    --log-format tqdm \
    --find-unused-parameters \
    --fp16 \
    --keep-last-epochs 10 \
    --patience 5 \
    --seed $SEED >${MODEL_DIR_STAGE1}/nohup.log 2>&1 &

wait

sunbo1999 commented 1 year ago

请问是把bart-model-file-from-transformers换成finetune_from_model吗

finetune_from_model: Optional[str] = field(
        default=None,
        metadata={
            "help": "finetune from a pretrained model; note that meters and lr scheduler will be reset"
        },
    )

HillZhang1999 commented 1 year ago

如果你本地的预训练模型是fairseq训的，把bart-model-file-from-transformers换成finetune_from_model就行；如果是hugging face训的，那么把bart-model-file-from-transformers的路径更改下就行

sunbo1999 commented 1 year ago

好的，感谢回复

HillZhang1999 / SynGEC

模型初始化 #15