Is there a way to have max-connections go beyond 100?

UKGovernmentBEIS / inspect_ai

Inspect: A framework for large language model evaluations

https://inspect.ai-safety-institute.org.uk/

MIT License

627 stars 118 forks source link

Is there a way to have max-connections go beyond 100? #847

Closed AarushSah closed 1 week ago

AarushSah commented 1 week ago

Is there a way to have max-connections go beyond 100? When I run an eval with --max-connections above 100, the amount of concurrent connections to the API always plateaus at 100.

jjallaire-aisi commented 1 week ago

This is likely to be a limit imposed by the HTTP client library not Inspect (we don't place any cap on connections outside of max-connections)