eole-nlp / eole

Open language modeling toolkit based on PyTorch
https://eole-nlp.github.io/eole
MIT License
58 stars 11 forks source link

Pydantic error with the toy example #81

Closed larsbun closed 2 months ago

larsbun commented 3 months ago

When I install eole from github (as in the last five minutes) exactly as specified in your documentations, I am able to successfully run eole build_vocab -config toy_en_de.yaml -n_sample 10000, but when I try eole train -config toy_en_de.yaml with the exact same contents in your docs, I get this:

$ eole train -config toy_en_de.yaml
Traceback (most recent call last):
  File "/work//envs/eole/bin/eole", line 33, in <module>
    sys.exit(load_entry_point('eole', 'console_scripts', 'eole')())
  File "/home//git-projects/eole/eole/bin/main.py", line 39, in main
    bin_cls.run(args)
  File "/home//git-projects/eole/eole/bin/run/train.py", line 68, in run
    config = cls.build_config(args)
  File "/home//git-projects/eole/eole/bin/run/__init__.py", line 42, in build_config
    config = cls.config_class(**config_dict)
  File "/work//envs/eole/lib/python3.10/site-packages/pydantic/main.py", line 193, in __init__
    self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 6 validation errors for TrainConfig
world_size
  Extra inputs are not permitted [type=extra_forbidden, input_value=1, input_type=int]
    For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
gpu_ranks
  Extra inputs are not permitted [type=extra_forbidden, input_value=[0], input_type=list]
    For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
model_path
  Extra inputs are not permitted [type=extra_forbidden, input_value='toy-ende/run/model', input_type=str]
    For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
save_checkpoint_steps
  Extra inputs are not permitted [type=extra_forbidden, input_value=500, input_type=int]
    For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
train_steps
  Extra inputs are not permitted [type=extra_forbidden, input_value=1000, input_type=int]
    For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
valid_steps
  Extra inputs are not permitted [type=extra_forbidden, input_value=500, input_type=int]
    For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden

I understand from searching that this is intended behavior by pydantic, but was unable to find the exact place in the eole source tree were these config options are confined.

francoishernandez commented 3 months ago

Thanks for reporting! I thought these docs had been updated, but it seems we missed some. We'll update soon. The settings causing issues here are now nested in the "training" section. If you have issues with other settings at some point, everything is defined under the eole.config module.

larsbun commented 2 months ago

Hi perhaps you could take a look at this again? I returned to the project now and get the same error and have to start digging again. A quickstart that actually runs verbatim would be very helpful. Thanks. https://eole-nlp.github.io/eole/docs/quickstart/

francoishernandez commented 2 months ago

Thanks for the follow-up, this went under the radar. Small fixes in #90, will rebuild the live docs soon.