benthosdev / benthos

Fancy stream processing made operationally mundane
https://www.benthos.dev
MIT License
7.68k stars 752 forks source link

`http` processors send body on `GET` #2527

Closed mskarbek closed 3 weeks ago

mskarbek commented 1 month ago

For:

pipeline:
  threads: -1
  processors:
    - branch:
        processors:
          - http:
              url: "https://REDACTED/service/rest/v1/components?repository=${! this.repo}"
              verb: GET
              headers:
                Accept: "application/json"
        result_map: "root.response = this"

I received 415 error from Sonatype Nexus server. Adding TRACE level of logging revealed that Benthos adds body to the request even if it is a GET which in turn upsets Nexus. Is there a way to avoid sending body with GET requests?

packman80 commented 4 weeks ago

From documentation:

A Bloblang mapping that describes how to create a request payload suitable for the child processors of this branch. If left empty then the branch will begin with an exact copy of the origin message (including metadata).

You need to get empty message without copy of original message. In this case you avoid sending body in GET request. Use it:

pipeline:
  threads: -1
  processors:
    - branch:
        request_map: root = "" 
        processors:
          - http:
              url: "https://REDACTED/service/rest/v1/components?repository=${! this.repo}"
              verb: GET
              headers:
                Accept: "application/json"
        result_map: "root.response = this"
packman80 commented 4 weeks ago

it may be worth clarifying this in the documentation. In addition, the option to ignore sending the request body in the GET method might be useful, even if the message (and the request body, respectively) is not empty. In some cases, this would simplify the configs

mihaitodor commented 4 weeks ago

You don't necessarily need to use a branch processor. If you only wish to retain the response of the http processor, then placing a mapping processor before it with root = "" will suffice.

it may be worth clarifying this in the documentation.

From https://www.benthos.dev/docs/components/processors/http:

Performs an HTTP request using a message batch as the request body

The branch processor is a common utility which can be combined with any other processor, depending on the use case. Not sure how to help users discover it easier. It's definitely covered in the cookbooks, like for example this one: https://www.benthos.dev/cookbooks/enrichments