Hi! This repo and the other one you have built using YARP are so helpful, thank you!
I have been researching how to load balance between AOAI instances when using Assistants API.
So far, it seems like we can use an affinity setting to send a cookie to the client and pinging the subsequent requests to the same backend.
Do you know if we can do something like that in your APIM smart load balancer, or the YARP one?
Really appreciate your thoughts and input.
Hi! This repo and the other one you have built using YARP are so helpful, thank you! I have been researching how to load balance between AOAI instances when using Assistants API. So far, it seems like we can use an affinity setting to send a cookie to the client and pinging the subsequent requests to the same backend. Do you know if we can do something like that in your APIM smart load balancer, or the YARP one? Really appreciate your thoughts and input.