search

awslabs / sockeye

Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch
https://awslabs.github.io/sockeye/
Apache License 2.0
1.21k stars 323 forks source link

Specif or omit --shared-vocab consistently when training and preparing the data error #1078

Closed NLP-Peter closed 1 year ago

NLP-Peter commented 1 year ago

Recently when I am trying to use sockeye3, it always returns an error "Specif or omit --shared-vocab consistently when training and preparing the data" while I am using the sockeye-prepare-data to prepare the data without specifying the shared-vocab. when using sockeye-train, the error shows up. I've confirmed that both the preparing and training procedure did not specify the shared-vocab. So is there anything I can do to fix this problem? or sockeye3 only support shared-vocab?

Best regards Peter

mjdenkowski commented 1 year ago

By default, sockeye-prepare-data does not use a shared vocabulary. The --shared-vocab argument turns on vocabulary sharing. This is required to train models that share/tie source and target weights.

Weight tying for sockeye-train is controlled with the --weight-tying-type argument. The default value of src_trg_softmax requires a shared vocabulary (automatically sets --shared-vocab). This requires the prepared data to also use --shared-vocab.

To train a model without weight tying, you can add --weight-tying-type none to the sockeye-train command. To train a model with weight tying, you can add --shared-vocab to the sockeye-prepare-data command.

NLP-Peter commented 1 year ago

thanks for your answer. It works for me.