Make sure eval subset is sampled without replacing

LAION-AI / Open-Assistant

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

https://open-assistant.io

Apache License 2.0

36.85k stars 3.22k forks source link

Make sure eval subset is sampled without replacing #3651

Closed andreaskoepf closed 11 months ago

andreaskoepf commented 11 months ago

Explicitly specify replace=False for numpy.random.choice (it was missing and default is replace=True which could lead to duplicate examples in the evaluation set).