Support automatically batching embeddings according to provider limit (e.g. Vonage supports 128 batch size
Motivation, pitch
Vonage supports 128 batch size
If i send batch size 129 it returns an error
If user on proxy sends 129, would make sense as an optional setting to batch the request automatically into two requests and return as one cohesive (as a feature that can be enabled)
The Feature
Support automatically batching embeddings according to provider limit (e.g. Vonage supports 128 batch size
Motivation, pitch
Vonage supports 128 batch size
If i send batch size 129 it returns an error
If user on proxy sends 129, would make sense as an optional setting to batch the request automatically into two requests and return as one cohesive (as a feature that can be enabled)
Twitter / LinkedIn details
cc: @PSU3D0