search

kedacore / http-add-on

Add-on for KEDA to scale HTTP workloads
https://kedacore.github.io/http-add-on/
Apache License 2.0
368 stars 96 forks source link

Support HTTPScaledObject scoped timeout #813

Open JorTurFer opened 1 year ago

JorTurFer commented 1 year ago

Proposal

Currently, there is a timeout configuration for setting the request timeout. Allowing HTTPScaledObjects to override the timeout for specific route can help to users who have some slow routes but they prefer to not increase the global timeout for all the routes just due to some of them

Use-Case

No response

Is this a feature you are interested in implementing yourself?

Maybe

Anything else?

No response

stale[bot] commented 10 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

stale[bot] commented 10 months ago

This issue has been automatically closed due to inactivity.

oto-macenauer-absa commented 8 months ago

Hi, is this expected in future releases?

We're using shared k8s cluster so it would be easier to use scoped timeout than asking platform team to come with values that fit all teams sharing the cluster. Also what kinds of timeouts is this expected to support?

JorTurFer commented 8 months ago

Hi, is this expected in future releases?

No, this has been closed automatically due to inactivity. Are you willing to contribute with it?

We're using shared k8s cluster so it would be easier to use scoped timeout than asking platform team to come with values that fit all teams sharing the cluster. Also what kinds of timeouts is this expected to support?

The original proposal was to override the "cold start timeout"

oto-macenauer-absa commented 8 months ago

No, this has been closed automatically due to inactivity. Are you willing to contribute with it?

Sorry, I don't have any experience with go and this doesn't seem like easy task to start.

The original proposal was to override the "cold start timeout"

Ok, we got API that can take long to answer so I was looking for something to override the responseHeaderTimeout

JorTurFer commented 8 months ago

I've checked and I guess that we could configure all the timeouts at same place from the HTTPScaledObject, so maybe could support all of them. https://github.com/kedacore/http-add-on/blob/main/interceptor/proxy_handlers.go#L46-L86

@tomkerkhove ?

tpfau commented 7 months ago

Having something like this would be great. We were trying to deploy LLMs this way on a cluster, having at most a few requests pending before adding new resources. But the responses (at least non streamed ones) can take quite some time before they are ready and those are then killed by the 500ms response header timeout and updating this for individual endpoints/pods would be great.

stale[bot] commented 4 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

tpfau commented 4 months ago

I would be willing to have a closer look at this but I have literally no idea where to start (and no experience whatsoever with coding for keda). Any suggestion or example of a variable that has similar behaviors where I could have a look to start it ?

harsh-viradia commented 3 months ago

I found a way to change the default timeout of keda http addon, while installing http addon via the helm we can change the default value from values.yaml file.

interceptor: responseHeaderTimeout: 300s tcpConnectTimeout: 300s keepAlive: 300s resources: limits: memory: 1Gi requests: memory: 512Mi

stale[bot] commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

stale[bot] commented 3 weeks ago

This issue has been automatically closed due to inactivity.