search

LAION-AI / Open-Assistant

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.
https://open-assistant.io
Apache License 2.0
36.94k stars 3.22k forks source link

OA dataloader raises an error when training the reward model #3588

Closed egg-west closed 1 year ago

egg-west commented 1 year ago

During training the reward model using the oasst dataset, using the following command:

python trainer_rm.py --configs defaults_rm oasst-rm-1-pythia-1.4b

the data loader raises an error: TypeError: 'NoneType' object is not iterable

│ Open-Assistant/model/model_training/custom_datasets/oasst_dataset.py:102 in leaf_filter│
│                                                                                        │
│    99   │   │   │   │   # their (ranked) replies as possible continuations.            │
│   100   │   │   │   │   return (                                                       │
│   101   │   │   │   │   │   thread[-1].role == "prompter"                              │
│ > 102   │   │   │   │   │   and len([r for r inthread[-1].replies if r.rank is not None]) > 1                     │
│   103   │   │   │   │   │   and thread_filter(thread)                                  │
│   104   │   │   │   │   )                                                              │
│   105   │   │   │   elif mode == "rl":                                                 │
╰────────────────────────────────────────────────────────────────────────────────────────╯

I inserted a condition in line 100 as below to fix this issue and submitted the referred PR

if thread[-1].replies is None:
    return False

PR: https://github.com/LAION-AI/Open-Assistant/pull/3587 With this bug fixed, I ran the training successfully.

Please take a look.