Add prevent-server-caching arg

Azure / azure-openai-benchmark

Azure OpenAI benchmarking tool

MIT License

130 stars 54 forks source link

Add prevent-server-caching arg #18

Closed michaeltremeer closed 6 months ago

michaeltremeer commented 10 months ago

Adds a prevent-server-caching arg in order to enable or disable the random prefix being added to all messages in each request. Useful for testing the effectiveness of any server-side caching on PTU resources.

technicianted commented 10 months ago

I think this PR should wait until we have a way to replay your own datasets otherwise it may not be very useful as we use the same prompt over and over currently.