Running eval requires openai key

allenai / open-instruct

Apache License 2.0

1.21k stars 166 forks source link

Running eval requires openai key #152

Closed natolambert closed 2 months ago

natolambert commented 5 months ago

Probably want to make this optional. Can close this if beaker secret just wasn't being loaded correctly, but not what I was expecting.

hamishivi commented 5 months ago

Yeah, we should make this optional. For evals that don't require an openai key, you can set OPENAI_API_KEY to any value and it should be fine.

hamishivi commented 2 months ago

Fixed with #135 - now a warning is simply logged. In the future, it probably would be good to add asserts to any tests that require openai keys so they error earlier rather than later.