search

ray-project / llmperf

LLMPerf is a library for validating and benchmarking LLMs
Apache License 2.0
470 stars 69 forks source link

Cleanup args handling #14

Closed SumanthRH closed 6 months ago

SumanthRH commented 7 months ago

Incremental PR to cleanup args handling in the main script. Have basically tried to address the TODO: " too much dependence on args globally. Clean up methods to not directly read from args to facilitate writing scripts."

I've also updated the openai dependency: Certain breaking API changes were introduced in the latest 1.0.0 release. openai.ChatCompletion errors out: 

You tried to access openai.ChatCompletion, but this is no longer supported in openai>=1.0.0 - see the README at https://github.com/openai/openai-python for the API.

You can run `openai migrate` to automatically upgrade your codebase to use the 1.0.0 interface. 

Alternatively, you can pin your installation to the old version, e.g. `pip install openai==0.28`

A detailed migration guide is available here: https://github.com/openai/openai-python/discussions/742

Atlernatively, I can run openai migrate and add changes to this PR itself, but would need to verify that all the OpenAI-compatible frameworks still work.

I'v tested the changes for gpt-3.5-turbo model. It should be fine for other frameworks as well.

cc @waleedkadous

kylehh commented 6 months ago

Merge for now but will be overwritten by LLMperf v2. Will move to LLMperf legacy repo