benthosdev / benthos

Fancy stream processing made operationally mundane
https://www.benthos.dev
MIT License
7.68k stars 752 forks source link

Add cookie handling for http client #2534

Open andreas-habel opened 3 weeks ago

andreas-habel commented 3 weeks ago

In one of our integrations, we have the challenge, that every call of our http output must contain a csrf token plus a set of cookies, because the receiving server is not truly stateless. Therefore we're calling a get token method via branch processor to receive the token and all necessary cookies, like a loadbalancer session id etc..

Unfortunately, Benthos currently does not accept multiple http headers with the same name properly, which is especially problematic for the Set-Cookie header.

This PR does not change the handling of headers in general, but it introduces a new Metadata field called Cookie which aggregates all incoming Set-Cookie headers in a ready-to-use Cookie value for subsequent calls.

Example Inbound:

Set-Cookie a=b; path=/, Httponly; Secure
Set-Cookie foo=bar; path=/

Resulting Metadata:

Cookie a=b; foo=bar

Since this would add a new metadata field whenever cookie headers are sent by the server, I'm not sure if this would need a separate config flag to disable this feature. Or if it even must be off by default?