axolotl-ai-cloud / axolotl

Go ahead and axolotl questions
https://axolotl-ai-cloud.github.io/axolotl/
Apache License 2.0
7.55k stars 816 forks source link

Pass `trust_remote_code` to `load_dataset(...)` for `datasets>=2.20.0` #1829

Open ccdv-ai opened 4 weeks ago

ccdv-ai commented 4 weeks ago

⚠️ Please check that this feature request hasn't been suggested before.

🔖 Feature description

datasets>=2.20.0 requires trust_remote_code=True to load datasets with a python loading script. This is usefull for custom and old datasets.

✔️ Solution

Pass trust_remote_code arg to load_dataset(...).

❓ Alternatives

No response

📝 Additional Context

No response

Acknowledgements