Make sure eval subset is sampled without replacing

LAION-AI / Open-Assistant

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

https://open-assistant.io

Apache License 2.0

37.1k stars 3.24k forks source link

Make sure eval subset is sampled without replacing #3651

Closed andreaskoepf closed 1 year ago

andreaskoepf commented 1 year ago

Explicitly specify replace=False for numpy.random.choice (it was missing and default is replace=True which could lead to duplicate examples in the evaluation set).