Closed RifleZhang closed 3 months ago
Hi @RifleZhang,
Sorry for the late response and thank you so much for your interest in our work!
We've recently updated the code. In the latest version, the Hugging Face dataset will be automatically downloaded and prepared for use. You can pull the latest code and follow the instructions provided here to start your training.
If you have any other questions, please don't hesitate to ask. We'll be more than happy to assist you!
https://github.com/RLHF-V/RLAIF-V/blob/main/muffin/data/data_processors.py#L97
The function is not loaded or defined. Also, gather_data_files_by_glob function may not match the parquet format of open-sourced data. Is it possible to update the dataset/dataloader for result replication? Thanks!