Closed HURIMOZ closed 1 month ago
Hello, This is expected as the configuration logic was fully revamped. We need to document this better. There are some nested sections to properly segregate the various scopes. See example here: https://eole-nlp.github.io/eole/docs/concepts/config Or in the various recipes https://github.com/eole-nlp/eole/tree/main/recipes
Please rewrite my config file so it can work with eole.
Please rewrite my config file so it can work with eole.
You can search through the actual code to find the correct config denominations. It took me around 10 mins, no big deal.
Hello, I have another validation error with my training config. I have remade the conversion. Do you have an idea ?
[2024-07-12 09:21:08,846 INFO] Loading checkpoint from models/llama3/llama3-8b-instruct
Traceback (most recent call last):
File "/usr/local/bin/eole", line 33, in <module>
sys.exit(load_entry_point('EOLE', 'console_scripts', 'eole')())
File "/workdir/eole/eole/bin/main.py", line 39, in main
bin_cls.run(args)
File "/workdir/eole/eole/bin/run/train.py", line 69, in run
train(config)
File "/workdir/eole/eole/bin/run/train.py", line 56, in train
train_process(config, device_id=0)
File "/workdir/eole/train_single.py", line 141, in main
checkpoint, vocabs, transforms, config = _init_train(config)
File "/workdir/eole/eole/train_single.py", line 67, in _init_train
checkpoint = load_checkpoint(config.training.train_from)
File "/workdir/eole/eole/models/model_saver.py", line 53, in load_checkpoint
_config = TrainConfig(**config_dict)
File "/usr/local/lib/python3.10/dist-packages/pydantic/main.py", line 176, in __init__
self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 13 validation errors for TrainConfig
model.transformer_lm.decoder.transformer_lm.num_kv
Extra inputs are not permitted [type=extra_forbidden, input_value=8, input_type=int]
For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.decoder.transformer_lm.pos_ffn_activation_fn
Extra inputs are not permitted [type=extra_forbidden, input_value='silu', input_type=str]
For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.decoder.transformer_lm.self_attn_type
Extra inputs are not permitted [type=extra_forbidden, input_value='scaled-dot-flash', input_type=str]
For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.decoder.transformer_lm.multiquery
Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool]
For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.decoder.transformer_lm.max_relative_positions
Extra inputs are not permitted [type=extra_forbidden, input_value=-1, input_type=int]
For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.model_type
Extra inputs are not permitted [type=extra_forbidden, input_value='text', input_type=str]
For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.num_kv
Extra inputs are not permitted [type=extra_forbidden, input_value=8, input_type=int]
For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.pos_ffn_activation_fn
Extra inputs are not permitted [type=extra_forbidden, input_value='silu', input_type=str]
For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.self_attn_type
Extra inputs are not permitted [type=extra_forbidden, input_value='scaled-dot-flash', input_type=str]
For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.multiquery
Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool]
For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.max_relative_positions
Extra inputs are not permitted [type=extra_forbidden, input_value=-1, input_type=int]
For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
training.model_dtype
Extra inputs are not permitted [type=extra_forbidden, input_value='fp16', input_type=str]
For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
data_task
Extra inputs are not permitted [type=extra_forbidden, input_value='lm', input_type=str]
For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
Providing the actual config which triggers the error would be helpful here.
That being said, the model.transformer_lm
prefix looks weird to me.
Do you have something like below in your config?
model:
transformer_lm:
x: ...
y: ...
If not that, there might be something wrong in the conversion/loading logic, but last time I checked it was not raising such issues.
EDIT: the model.transformer_lm
seems normal in the context of pydantic. The issue is that the parameters in question are not supported anymore.
this looks like an old config field
pos_ffn_activation_fn
has been renamed some time ago
also bear in mind that each PR might change some option names
for instance max_relative_positions
is no longer used see the PR re: position_encoding
@HURIMOZ see #69, I quickly put together a small conversion script to facilitate transitioning from OpenNMT-py.
Hi everyone, can you please provide a more robust template, instead of the very old WMT17 config example? Iʻd like to use pretrained subword embeddings, Iʻd like to use tokenization models created with standalone sentencepiece, etc. WMT17 is so outdated.
@HURIMOZ similar answer as here -- https://github.com/eole-nlp/eole/issues/61#issuecomment-2219877707 In the meantime, you can probably take some inspiration from the docs and other recipes. I updated some doc in #74, might be helpful.
Iʻm using my previous OpenNMT-py config file to build a bilingual translation model using the transformer but I get a rife of trouble:
(TY-EN) ubuntu@ip-172-31-2-199:~/TY-EN/eole/recipes/data$ eole train -config config.yaml Traceback (most recent call last): File "/home/ubuntu/TY-EN/TY-EN/bin/eole", line 33, in <module> sys.exit(load_entry_point('eole', 'console_scripts', 'eole')()) File "/home/ubuntu/TY-EN/eole/eole/bin/main.py", line 39, in main bin_cls.run(args) File "/home/ubuntu/TY-EN/eole/eole/bin/run/train.py", line 68, in run config = cls.build_config(args) File "/home/ubuntu/TY-EN/eole/eole/bin/run/__init__.py", line 42, in build_config config = cls.config_class(**config_dict) File "/home/ubuntu/TY-EN/TY-EN/lib/python3.10/site-packages/pydantic/main.py", line 193, in __init__ self.__pydantic_validator__.validate_python(data, self_instance=self) pydantic_core._pydantic_core.ValidationError: 52 validation errors for TrainConfig data.corpus_1.src_lang Extra inputs are not permitted [type=extra_forbidden, input_value='fr', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.corpus_1.tgt_lang Extra inputs are not permitted [type=extra_forbidden, input_value='ty', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.corpus_1.norm_quote_commas Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.corpus_1.norm_numbers Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.src_lang Extra inputs are not permitted [type=extra_forbidden, input_value='fr', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.tgt_lang Extra inputs are not permitted [type=extra_forbidden, input_value='ty', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.norm_quote_commas Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.norm_numbers Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden encoder_type Extra inputs are not permitted [type=extra_forbidden, input_value='transformer', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden decoder_type Extra inputs are not permitted [type=extra_forbidden, input_value='transformer', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden position_encoding Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden layers Extra inputs are not permitted [type=extra_forbidden, input_value=6, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden hidden_size Extra inputs are not permitted [type=extra_forbidden, input_value=256, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden heads Extra inputs are not permitted [type=extra_forbidden, input_value=8, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden transformer_ff Extra inputs are not permitted [type=extra_forbidden, input_value=2048, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden dropout_steps Extra inputs are not permitted [type=extra_forbidden, input_value=[0], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden dropout Extra inputs are not permitted [type=extra_forbidden, input_value=[0.3], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden attention_dropout Extra inputs are not permitted [type=extra_forbidden, input_value=[0.3], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden lora_dropout Extra inputs are not permitted [type=extra_forbidden, input_value=0.3, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden word_vec_size Extra inputs are not permitted [type=extra_forbidden, input_value=256, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden model_dtype Extra inputs are not permitted [type=extra_forbidden, input_value='fp16', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden optim Extra inputs are not permitted [type=extra_forbidden, input_value='adam', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden adam_beta1 Extra inputs are not permitted [type=extra_forbidden, input_value=0.9, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden adam_beta2 Extra inputs are not permitted [type=extra_forbidden, input_value=0.998, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden decay_method Extra inputs are not permitted [type=extra_forbidden, input_value='noam', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden learning_rate Extra inputs are not permitted [type=extra_forbidden, input_value=2, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden max_grad_norm Extra inputs are not permitted [type=extra_forbidden, input_value=0.0, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden normalization Extra inputs are not permitted [type=extra_forbidden, input_value='tokens', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden param_init Extra inputs are not permitted [type=extra_forbidden, input_value=0.0, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden param_init_glorot Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden max_relative_positions Extra inputs are not permitted [type=extra_forbidden, input_value=-1, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden batch_size Extra inputs are not permitted [type=extra_forbidden, input_value=2048, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden batch_type Extra inputs are not permitted [type=extra_forbidden, input_value='tokens', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden accum_count Extra inputs are not permitted [type=extra_forbidden, input_value=8, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden max_generator_batches Extra inputs are not permitted [type=extra_forbidden, input_value=2, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden src_subword_type Extra inputs are not permitted [type=extra_forbidden, input_value='sentencepiece', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden tgt_subword_type Extra inputs are not permitted [type=extra_forbidden, input_value='sentencepiece', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden src_subword_model Extra inputs are not permitted [type=extra_forbidden, input_value='fr.wiki.bpe.vs10000.model', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden tgt_subword_model Extra inputs are not permitted [type=extra_forbidden, input_value='tgt_spm.model', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden save_model Extra inputs are not permitted [type=extra_forbidden, input_value='run/model', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden keep_checkpoint Extra inputs are not permitted [type=extra_forbidden, input_value=20, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden save_checkpoint_steps Extra inputs are not permitted [type=extra_forbidden, input_value=1000, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden train_steps Extra inputs are not permitted [type=extra_forbidden, input_value=100000, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden valid_steps Extra inputs are not permitted [type=extra_forbidden, input_value=500, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden warmup_steps Extra inputs are not permitted [type=extra_forbidden, input_value=8000, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden early_stopping Extra inputs are not permitted [type=extra_forbidden, input_value=5, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden early_stopping_criteria Extra inputs are not permitted [type=extra_forbidden, input_value='accuracy', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden on_error Extra inputs are not permitted [type=extra_forbidden, input_value='raise', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden world_size Extra inputs are not permitted [type=extra_forbidden, input_value=1, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden gpu_ranks Extra inputs are not permitted [type=extra_forbidden, input_value=[0], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden subword_nbest Extra inputs are not permitted [type=extra_forbidden, input_value=20, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden subword_alpha Extra inputs are not permitted [type=extra_forbidden, input_value=0.1, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden (TY-EN) ubuntu@ip-172-31-2-199:~/TY-EN/eole/recipes/data$ eole train -config config.yaml Traceback (most recent call last): File "/home/ubuntu/TY-EN/TY-EN/bin/eole", line 33, in <module> sys.exit(load_entry_point('eole', 'console_scripts', 'eole')()) File "/home/ubuntu/TY-EN/eole/eole/bin/main.py", line 39, in main bin_cls.run(args) File "/home/ubuntu/TY-EN/eole/eole/bin/run/train.py", line 68, in run config = cls.build_config(args) File "/home/ubuntu/TY-EN/eole/eole/bin/run/__init__.py", line 42, in build_config config = cls.config_class(**config_dict) File "/home/ubuntu/TY-EN/TY-EN/lib/python3.10/site-packages/pydantic/main.py", line 193, in __init__ self.__pydantic_validator__.validate_python(data, self_instance=self) pydantic_core._pydantic_core.ValidationError: 46 validation errors for TrainConfig data.corpus_1.src_lang Extra inputs are not permitted [type=extra_forbidden, input_value='fr', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.corpus_1.tgt_lang Extra inputs are not permitted [type=extra_forbidden, input_value='ty', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.corpus_1.norm_quote_commas Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.corpus_1.norm_numbers Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.src_lang Extra inputs are not permitted [type=extra_forbidden, input_value='fr', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.tgt_lang Extra inputs are not permitted [type=extra_forbidden, input_value='ty', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.norm_quote_commas Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.norm_numbers Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden dropout_steps Extra inputs are not permitted [type=extra_forbidden, input_value=[0], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden dropout Extra inputs are not permitted [type=extra_forbidden, input_value=[0.3], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden attention_dropout Extra inputs are not permitted [type=extra_forbidden, input_value=[0.3], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden lora_dropout Extra inputs are not permitted [type=extra_forbidden, input_value=0.3, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden word_vec_size Extra inputs are not permitted [type=extra_forbidden, input_value=256, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden model_dtype Extra inputs are not permitted [type=extra_forbidden, input_value='fp16', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden optim Extra inputs are not permitted [type=extra_forbidden, input_value='adam', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden adam_beta1 Extra inputs are not permitted [type=extra_forbidden, input_value=0.9, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden adam_beta2 Extra inputs are not permitted [type=extra_forbidden, input_value=0.998, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden decay_method Extra inputs are not permitted [type=extra_forbidden, input_value='noam', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden learning_rate Extra inputs are not permitted [type=extra_forbidden, input_value=2, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden max_grad_norm Extra inputs are not permitted [type=extra_forbidden, input_value=0.0, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden normalization Extra inputs are not permitted [type=extra_forbidden, input_value='tokens', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden param_init Extra inputs are not permitted [type=extra_forbidden, input_value=0.0, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden param_init_glorot Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden position_encoding Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden max_relative_positions Extra inputs are not permitted [type=extra_forbidden, input_value=-1, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden batch_size Extra inputs are not permitted [type=extra_forbidden, input_value=2048, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden batch_type Extra inputs are not permitted [type=extra_forbidden, input_value='tokens', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden accum_count Extra inputs are not permitted [type=extra_forbidden, input_value=8, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden max_generator_batches Extra inputs are not permitted [type=extra_forbidden, input_value=2, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden src_subword_type Extra inputs are not permitted [type=extra_forbidden, input_value='sentencepiece', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden tgt_subword_type Extra inputs are not permitted [type=extra_forbidden, input_value='sentencepiece', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden src_subword_model Extra inputs are not permitted [type=extra_forbidden, input_value='fr.wiki.bpe.vs10000.model', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden tgt_subword_model Extra inputs are not permitted [type=extra_forbidden, input_value='tgt_spm.model', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden save_model Extra inputs are not permitted [type=extra_forbidden, input_value='run/model', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden keep_checkpoint Extra inputs are not permitted [type=extra_forbidden, input_value=20, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden save_checkpoint_steps Extra inputs are not permitted [type=extra_forbidden, input_value=1000, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden train_steps Extra inputs are not permitted [type=extra_forbidden, input_value=100000, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden valid_steps Extra inputs are not permitted [type=extra_forbidden, input_value=500, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden warmup_steps Extra inputs are not permitted [type=extra_forbidden, input_value=8000, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden early_stopping Extra inputs are not permitted [type=extra_forbidden, input_value=5, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden early_stopping_criteria Extra inputs are not permitted [type=extra_forbidden, input_value='accuracy', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden on_error Extra inputs are not permitted [type=extra_forbidden, input_value='raise', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden world_size Extra inputs are not permitted [type=extra_forbidden, input_value=1, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden gpu_ranks Extra inputs are not permitted [type=extra_forbidden, input_value=[0], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden subword_nbest Extra inputs are not permitted [type=extra_forbidden, input_value=20, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden subword_alpha Extra inputs are not permitted [type=extra_forbidden, input_value=0.1, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
My config file: `# Configuration file for OpenNMT-py training and translationModel architecture configuration
encoder_type: transformer decoder_type: transformer position_encoding: true layers: 6 hidden_size: 256 heads: 8 transformer_ff: 2048 dropout_steps: [0] dropout: [0.3] attention_dropout: [0.3] lora_dropout: 0.3
Pretrained embeddings configuration for the source language
src_embeddings: fr.wiki.bpe.vs10000.d300.w2v-256.txt # Ensure this path is correct embeddings_type: word2vec # Ensure this matches the format of your embeddings word_vec_size: 256
Optimization
model_dtype: "fp16" optim: adam adam_beta1: 0.9 adam_beta2: 0.998 decay_method: noam learning_rate: 2 max_grad_norm: 0.0 normalization: tokens param_init: 0.0 param_init_glorot: true position_encoding: false max_relative_positions: -1
Batching
batch_size: 2048 batch_type: tokens accum_count: 8 max_generator_batches: 2
Tokenization options
src_subword_type: sentencepiece # Specify the tokenization method for the source side tgt_subword_type: sentencepiece # Specify the tokenization method for the target side src_subword_model: fr.wiki.bpe.vs10000.model # Path to the BPEmb model tgt_subword_model: tgt_spm.model # Path to the SentencePiece model src_vocab: fr.wiki.bpe.vs10000.src # Path to the source vocabulary tgt_vocab: tgt_spm.tgt # Path to the target vocabulary
Training hyperparameters
save_model: run/model keep_checkpoint: 20 save_checkpoint_steps: 1000 seed: -1 train_steps: 100000 valid_steps: 500 warmup_steps: 8000 report_every: 500 early_stopping: 5 early_stopping_criteria: accuracy
TensorBoard configuration
tensorboard: true tensorboard_log_dir: run/logs
Error handling
on_error: raise
Train on a single GPU
world_size: 1 gpu_ranks: [0]
Path for saving data required by pretrained embeddings
save_data: processed
Number of candidates for SentencePiece sampling
subword_nbest: 20
Smoothing parameter for SentencePiece sampling
subword_alpha: 0.1
Specific arguments for pyonmttok
src_onmttok_kwargs: "{'mode': 'none', 'spacer_annotate': True}"
tgt_onmttok_kwargs: "{'mode': 'none', 'spacer_annotate': True}"
Corpus opts:
data: corpus_1: path_src: src-train.fr path_tgt: tgt-train.ty transforms: [normalize, sentencepiece, filtertoolong] weight: 1 src_lang: fr tgt_lang: ty norm_quote_commas: true norm_numbers: true valid: path_src: src-val.fr path_tgt: tgt-val.ty transforms: [normalize, sentencepiece, filtertoolong] src_lang: fr tgt_lang: ty norm_quote_commas: true norm_numbers: true`