search

eole-nlp / eole

Open language modeling toolkit based on PyTorch
https://eole-nlp.github.io/eole
MIT License
35 stars 8 forks source link

Extra inputs are not permitted - Pydantic errors #66

Closed HURIMOZ closed 1 month ago

HURIMOZ commented 1 month ago

Iʻm using my previous OpenNMT-py config file to build a bilingual translation model using the transformer but I get a rife of trouble: (TY-EN) ubuntu@ip-172-31-2-199:~/TY-EN/eole/recipes/data$ eole train -config config.yaml Traceback (most recent call last): File "/home/ubuntu/TY-EN/TY-EN/bin/eole", line 33, in <module> sys.exit(load_entry_point('eole', 'console_scripts', 'eole')()) File "/home/ubuntu/TY-EN/eole/eole/bin/main.py", line 39, in main bin_cls.run(args) File "/home/ubuntu/TY-EN/eole/eole/bin/run/train.py", line 68, in run config = cls.build_config(args) File "/home/ubuntu/TY-EN/eole/eole/bin/run/__init__.py", line 42, in build_config config = cls.config_class(**config_dict) File "/home/ubuntu/TY-EN/TY-EN/lib/python3.10/site-packages/pydantic/main.py", line 193, in __init__ self.__pydantic_validator__.validate_python(data, self_instance=self) pydantic_core._pydantic_core.ValidationError: 52 validation errors for TrainConfig data.corpus_1.src_lang Extra inputs are not permitted [type=extra_forbidden, input_value='fr', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.corpus_1.tgt_lang Extra inputs are not permitted [type=extra_forbidden, input_value='ty', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.corpus_1.norm_quote_commas Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.corpus_1.norm_numbers Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.src_lang Extra inputs are not permitted [type=extra_forbidden, input_value='fr', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.tgt_lang Extra inputs are not permitted [type=extra_forbidden, input_value='ty', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.norm_quote_commas Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.norm_numbers Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden encoder_type Extra inputs are not permitted [type=extra_forbidden, input_value='transformer', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden decoder_type Extra inputs are not permitted [type=extra_forbidden, input_value='transformer', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden position_encoding Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden layers Extra inputs are not permitted [type=extra_forbidden, input_value=6, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden hidden_size Extra inputs are not permitted [type=extra_forbidden, input_value=256, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden heads Extra inputs are not permitted [type=extra_forbidden, input_value=8, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden transformer_ff Extra inputs are not permitted [type=extra_forbidden, input_value=2048, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden dropout_steps Extra inputs are not permitted [type=extra_forbidden, input_value=[0], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden dropout Extra inputs are not permitted [type=extra_forbidden, input_value=[0.3], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden attention_dropout Extra inputs are not permitted [type=extra_forbidden, input_value=[0.3], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden lora_dropout Extra inputs are not permitted [type=extra_forbidden, input_value=0.3, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden word_vec_size Extra inputs are not permitted [type=extra_forbidden, input_value=256, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden model_dtype Extra inputs are not permitted [type=extra_forbidden, input_value='fp16', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden optim Extra inputs are not permitted [type=extra_forbidden, input_value='adam', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden adam_beta1 Extra inputs are not permitted [type=extra_forbidden, input_value=0.9, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden adam_beta2 Extra inputs are not permitted [type=extra_forbidden, input_value=0.998, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden decay_method Extra inputs are not permitted [type=extra_forbidden, input_value='noam', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden learning_rate Extra inputs are not permitted [type=extra_forbidden, input_value=2, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden max_grad_norm Extra inputs are not permitted [type=extra_forbidden, input_value=0.0, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden normalization Extra inputs are not permitted [type=extra_forbidden, input_value='tokens', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden param_init Extra inputs are not permitted [type=extra_forbidden, input_value=0.0, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden param_init_glorot Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden max_relative_positions Extra inputs are not permitted [type=extra_forbidden, input_value=-1, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden batch_size Extra inputs are not permitted [type=extra_forbidden, input_value=2048, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden batch_type Extra inputs are not permitted [type=extra_forbidden, input_value='tokens', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden accum_count Extra inputs are not permitted [type=extra_forbidden, input_value=8, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden max_generator_batches Extra inputs are not permitted [type=extra_forbidden, input_value=2, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden src_subword_type Extra inputs are not permitted [type=extra_forbidden, input_value='sentencepiece', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden tgt_subword_type Extra inputs are not permitted [type=extra_forbidden, input_value='sentencepiece', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden src_subword_model Extra inputs are not permitted [type=extra_forbidden, input_value='fr.wiki.bpe.vs10000.model', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden tgt_subword_model Extra inputs are not permitted [type=extra_forbidden, input_value='tgt_spm.model', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden save_model Extra inputs are not permitted [type=extra_forbidden, input_value='run/model', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden keep_checkpoint Extra inputs are not permitted [type=extra_forbidden, input_value=20, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden save_checkpoint_steps Extra inputs are not permitted [type=extra_forbidden, input_value=1000, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden train_steps Extra inputs are not permitted [type=extra_forbidden, input_value=100000, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden valid_steps Extra inputs are not permitted [type=extra_forbidden, input_value=500, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden warmup_steps Extra inputs are not permitted [type=extra_forbidden, input_value=8000, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden early_stopping Extra inputs are not permitted [type=extra_forbidden, input_value=5, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden early_stopping_criteria Extra inputs are not permitted [type=extra_forbidden, input_value='accuracy', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden on_error Extra inputs are not permitted [type=extra_forbidden, input_value='raise', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden world_size Extra inputs are not permitted [type=extra_forbidden, input_value=1, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden gpu_ranks Extra inputs are not permitted [type=extra_forbidden, input_value=[0], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden subword_nbest Extra inputs are not permitted [type=extra_forbidden, input_value=20, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden subword_alpha Extra inputs are not permitted [type=extra_forbidden, input_value=0.1, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden (TY-EN) ubuntu@ip-172-31-2-199:~/TY-EN/eole/recipes/data$ eole train -config config.yaml Traceback (most recent call last): File "/home/ubuntu/TY-EN/TY-EN/bin/eole", line 33, in <module> sys.exit(load_entry_point('eole', 'console_scripts', 'eole')()) File "/home/ubuntu/TY-EN/eole/eole/bin/main.py", line 39, in main bin_cls.run(args) File "/home/ubuntu/TY-EN/eole/eole/bin/run/train.py", line 68, in run config = cls.build_config(args) File "/home/ubuntu/TY-EN/eole/eole/bin/run/__init__.py", line 42, in build_config config = cls.config_class(**config_dict) File "/home/ubuntu/TY-EN/TY-EN/lib/python3.10/site-packages/pydantic/main.py", line 193, in __init__ self.__pydantic_validator__.validate_python(data, self_instance=self) pydantic_core._pydantic_core.ValidationError: 46 validation errors for TrainConfig data.corpus_1.src_lang Extra inputs are not permitted [type=extra_forbidden, input_value='fr', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.corpus_1.tgt_lang Extra inputs are not permitted [type=extra_forbidden, input_value='ty', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.corpus_1.norm_quote_commas Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.corpus_1.norm_numbers Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.src_lang Extra inputs are not permitted [type=extra_forbidden, input_value='fr', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.tgt_lang Extra inputs are not permitted [type=extra_forbidden, input_value='ty', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.norm_quote_commas Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden data.valid.norm_numbers Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden dropout_steps Extra inputs are not permitted [type=extra_forbidden, input_value=[0], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden dropout Extra inputs are not permitted [type=extra_forbidden, input_value=[0.3], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden attention_dropout Extra inputs are not permitted [type=extra_forbidden, input_value=[0.3], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden lora_dropout Extra inputs are not permitted [type=extra_forbidden, input_value=0.3, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden word_vec_size Extra inputs are not permitted [type=extra_forbidden, input_value=256, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden model_dtype Extra inputs are not permitted [type=extra_forbidden, input_value='fp16', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden optim Extra inputs are not permitted [type=extra_forbidden, input_value='adam', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden adam_beta1 Extra inputs are not permitted [type=extra_forbidden, input_value=0.9, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden adam_beta2 Extra inputs are not permitted [type=extra_forbidden, input_value=0.998, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden decay_method Extra inputs are not permitted [type=extra_forbidden, input_value='noam', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden learning_rate Extra inputs are not permitted [type=extra_forbidden, input_value=2, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden max_grad_norm Extra inputs are not permitted [type=extra_forbidden, input_value=0.0, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden normalization Extra inputs are not permitted [type=extra_forbidden, input_value='tokens', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden param_init Extra inputs are not permitted [type=extra_forbidden, input_value=0.0, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden param_init_glorot Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden position_encoding Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden max_relative_positions Extra inputs are not permitted [type=extra_forbidden, input_value=-1, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden batch_size Extra inputs are not permitted [type=extra_forbidden, input_value=2048, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden batch_type Extra inputs are not permitted [type=extra_forbidden, input_value='tokens', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden accum_count Extra inputs are not permitted [type=extra_forbidden, input_value=8, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden max_generator_batches Extra inputs are not permitted [type=extra_forbidden, input_value=2, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden src_subword_type Extra inputs are not permitted [type=extra_forbidden, input_value='sentencepiece', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden tgt_subword_type Extra inputs are not permitted [type=extra_forbidden, input_value='sentencepiece', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden src_subword_model Extra inputs are not permitted [type=extra_forbidden, input_value='fr.wiki.bpe.vs10000.model', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden tgt_subword_model Extra inputs are not permitted [type=extra_forbidden, input_value='tgt_spm.model', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden save_model Extra inputs are not permitted [type=extra_forbidden, input_value='run/model', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden keep_checkpoint Extra inputs are not permitted [type=extra_forbidden, input_value=20, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden save_checkpoint_steps Extra inputs are not permitted [type=extra_forbidden, input_value=1000, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden train_steps Extra inputs are not permitted [type=extra_forbidden, input_value=100000, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden valid_steps Extra inputs are not permitted [type=extra_forbidden, input_value=500, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden warmup_steps Extra inputs are not permitted [type=extra_forbidden, input_value=8000, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden early_stopping Extra inputs are not permitted [type=extra_forbidden, input_value=5, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden early_stopping_criteria Extra inputs are not permitted [type=extra_forbidden, input_value='accuracy', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden on_error Extra inputs are not permitted [type=extra_forbidden, input_value='raise', input_type=str] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden world_size Extra inputs are not permitted [type=extra_forbidden, input_value=1, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden gpu_ranks Extra inputs are not permitted [type=extra_forbidden, input_value=[0], input_type=list] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden subword_nbest Extra inputs are not permitted [type=extra_forbidden, input_value=20, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden subword_alpha Extra inputs are not permitted [type=extra_forbidden, input_value=0.1, input_type=float] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden My config file: `# Configuration file for OpenNMT-py training and translation

Model architecture configuration

encoder_type: transformer decoder_type: transformer position_encoding: true layers: 6 hidden_size: 256 heads: 8 transformer_ff: 2048 dropout_steps: [0] dropout: [0.3] attention_dropout: [0.3] lora_dropout: 0.3

Pretrained embeddings configuration for the source language

src_embeddings: fr.wiki.bpe.vs10000.d300.w2v-256.txt # Ensure this path is correct embeddings_type: word2vec # Ensure this matches the format of your embeddings word_vec_size: 256

Optimization

model_dtype: "fp16" optim: adam adam_beta1: 0.9 adam_beta2: 0.998 decay_method: noam learning_rate: 2 max_grad_norm: 0.0 normalization: tokens param_init: 0.0 param_init_glorot: true position_encoding: false max_relative_positions: -1

Batching

batch_size: 2048 batch_type: tokens accum_count: 8 max_generator_batches: 2

Tokenization options

src_subword_type: sentencepiece # Specify the tokenization method for the source side tgt_subword_type: sentencepiece # Specify the tokenization method for the target side src_subword_model: fr.wiki.bpe.vs10000.model # Path to the BPEmb model tgt_subword_model: tgt_spm.model # Path to the SentencePiece model src_vocab: fr.wiki.bpe.vs10000.src # Path to the source vocabulary tgt_vocab: tgt_spm.tgt # Path to the target vocabulary

Training hyperparameters

save_model: run/model keep_checkpoint: 20 save_checkpoint_steps: 1000 seed: -1 train_steps: 100000 valid_steps: 500 warmup_steps: 8000 report_every: 500 early_stopping: 5 early_stopping_criteria: accuracy

TensorBoard configuration

tensorboard: true tensorboard_log_dir: run/logs

Error handling

on_error: raise

Train on a single GPU

world_size: 1 gpu_ranks: [0]

Path for saving data required by pretrained embeddings

save_data: processed

Number of candidates for SentencePiece sampling

subword_nbest: 20

Smoothing parameter for SentencePiece sampling

subword_alpha: 0.1

Specific arguments for pyonmttok

src_onmttok_kwargs: "{'mode': 'none', 'spacer_annotate': True}"

tgt_onmttok_kwargs: "{'mode': 'none', 'spacer_annotate': True}"

Corpus opts:

data: corpus_1: path_src: src-train.fr path_tgt: tgt-train.ty transforms: [normalize, sentencepiece, filtertoolong] weight: 1 src_lang: fr tgt_lang: ty norm_quote_commas: true norm_numbers: true valid: path_src: src-val.fr path_tgt: tgt-val.ty transforms: [normalize, sentencepiece, filtertoolong] src_lang: fr tgt_lang: ty norm_quote_commas: true norm_numbers: true`

francoishernandez commented 1 month ago

Hello, This is expected as the configuration logic was fully revamped. We need to document this better. There are some nested sections to properly segregate the various scopes. See example here: https://eole-nlp.github.io/eole/docs/concepts/config Or in the various recipes https://github.com/eole-nlp/eole/tree/main/recipes

HURIMOZ commented 1 month ago

Please rewrite my config file so it can work with eole.

ArtanisTheOne commented 1 month ago

Please rewrite my config file so it can work with eole.

You can search through the actual code to find the correct config denominations. It took me around 10 mins, no big deal.

l-k-11235 commented 1 month ago

Hello, I have another validation error with my training config. I have remade the conversion. Do you have an idea ?

[2024-07-12 09:21:08,846 INFO] Loading checkpoint from models/llama3/llama3-8b-instruct
Traceback (most recent call last):
  File "/usr/local/bin/eole", line 33, in <module>
    sys.exit(load_entry_point('EOLE', 'console_scripts', 'eole')())
  File "/workdir/eole/eole/bin/main.py", line 39, in main
    bin_cls.run(args)
  File "/workdir/eole/eole/bin/run/train.py", line 69, in run
    train(config)
  File "/workdir/eole/eole/bin/run/train.py", line 56, in train
    train_process(config, device_id=0)
  File "/workdir/eole/train_single.py", line 141, in main
    checkpoint, vocabs, transforms, config = _init_train(config)
  File "/workdir/eole/eole/train_single.py", line 67, in _init_train
    checkpoint = load_checkpoint(config.training.train_from)
  File "/workdir/eole/eole/models/model_saver.py", line 53, in load_checkpoint
    _config = TrainConfig(**config_dict)
  File "/usr/local/lib/python3.10/dist-packages/pydantic/main.py", line 176, in __init__
    self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 13 validation errors for TrainConfig
model.transformer_lm.decoder.transformer_lm.num_kv
  Extra inputs are not permitted [type=extra_forbidden, input_value=8, input_type=int]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.decoder.transformer_lm.pos_ffn_activation_fn
  Extra inputs are not permitted [type=extra_forbidden, input_value='silu', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.decoder.transformer_lm.self_attn_type
  Extra inputs are not permitted [type=extra_forbidden, input_value='scaled-dot-flash', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.decoder.transformer_lm.multiquery
  Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.decoder.transformer_lm.max_relative_positions
  Extra inputs are not permitted [type=extra_forbidden, input_value=-1, input_type=int]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.model_type
  Extra inputs are not permitted [type=extra_forbidden, input_value='text', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.num_kv
  Extra inputs are not permitted [type=extra_forbidden, input_value=8, input_type=int]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.pos_ffn_activation_fn
  Extra inputs are not permitted [type=extra_forbidden, input_value='silu', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.self_attn_type
  Extra inputs are not permitted [type=extra_forbidden, input_value='scaled-dot-flash', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.multiquery
  Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
model.transformer_lm.max_relative_positions
  Extra inputs are not permitted [type=extra_forbidden, input_value=-1, input_type=int]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
training.model_dtype
  Extra inputs are not permitted [type=extra_forbidden, input_value='fp16', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
data_task
  Extra inputs are not permitted [type=extra_forbidden, input_value='lm', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden
francoishernandez commented 1 month ago

Providing the actual config which triggers the error would be helpful here. That being said, the model.transformer_lmprefix looks weird to me. Do you have something like below in your config?

model:
  transformer_lm:
    x: ...
    y: ...

If not that, there might be something wrong in the conversion/loading logic, but last time I checked it was not raising such issues.

EDIT: the model.transformer_lm seems normal in the context of pydantic. The issue is that the parameters in question are not supported anymore.

vince62s commented 1 month ago

this looks like an old config field pos_ffn_activation_fn has been renamed some time ago

also bear in mind that each PR might change some option names for instance max_relative_positions is no longer used see the PR re: position_encoding

francoishernandez commented 1 month ago

@HURIMOZ see #69, I quickly put together a small conversion script to facilitate transitioning from OpenNMT-py.

HURIMOZ commented 1 month ago

Hi everyone, can you please provide a more robust template, instead of the very old WMT17 config example? Iʻd like to use pretrained subword embeddings, Iʻd like to use tokenization models created with standalone sentencepiece, etc. WMT17 is so outdated.

francoishernandez commented 1 month ago

@HURIMOZ similar answer as here -- https://github.com/eole-nlp/eole/issues/61#issuecomment-2219877707 In the meantime, you can probably take some inspiration from the docs and other recipes. I updated some doc in #74, might be helpful.