eole-nlp / eole

Open language modeling toolkit based on PyTorch
https://eole-nlp.github.io/eole
MIT License
53 stars 11 forks source link

Can't follow recipe #48

Closed Kai-Piontek closed 3 months ago

Kai-Piontek commented 3 months ago

Dear Eole Community

I am trying to follow the recipe for building the wmt17 ende transformer, but I get stuck here: My installation is via docker.

eole build_vocab --config wmt17_ende.yaml --n_sample -1

Traceback (most recent call last): File "/usr/local/bin/eole", line 33, in sys.exit(load_entry_point('EOLE', 'console_scripts', 'eole')()) File "/var/transformer/data/eole/eole/eole/bin/main.py", line 39, in main bin_cls.run(args) File "/var/transformer/data/eole/eole/eole/bin/run/build_vocab.py", line 272, in run config = cls.build_config(args) File "/var/transformer/data/eole/eole/eole/bin/run/init.py", line 42, in build_config config = cls.config_class(**config_dict) File "/usr/local/lib/python3.10/dist-packages/pydantic/main.py", line 176, in init self.__pydantic_validator__.validate_python(data, self_instance=self) pydantic_core._pydantic_core.ValidationError: 4 validation errors for BuildVocabConfig report_every Extra inputs are not permitted [type=extra_forbidden, input_value=100, input_type=int] For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden valid_metrics Extra inputs are not permitted [type=extra_forbidden, input_value=['BLEU'], input_type=list] For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden training Extra inputs are not permitted [type=extra_forbidden, input_value={'model_path': 'wmt17en...ormalization': 'tokens'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden model Extra inputs are not permitted [type=extra_forbidden, input_value={'architecture': 'transfo...sition_encoding': True}}, input_type=dict] For further information visit https://errors.pydantic.dev/2.7/v/extra_forbidden

What am I doing wrong? Best Kai

francoishernandez commented 3 months ago

Hi there, Thanks for reporting.

49 should fix your issue.

For context, initially we were more permissive on the various config fields, but we switched to enforce more structure, and the build_vocab logic had not been adapted in that sense.

francoishernandez commented 3 months ago

@Kai-Piontek closing this as it should have been resolved by the aforementioned PR. Feel free to ask new questions if needed!