search

spring-cloud / stream-applications

Functions and Spring Cloud Stream Applications for data driven microservices
https://spring.io/projects/spring-cloud-stream-applications
253 stars 104 forks source link

delete all headers content using spelFunction in http-source #442

Closed gauravsingh96 closed 1 year ago

gauravsingh96 commented 1 year ago

Hi Team,

is there a way to clear entire header content using the spelFunction ? I am trying below definition but it gives error - org.springframework.messaging.MessageHandlingException: Expression evaluation failed: headers=null; nested exception is org.springframework.expression.spel.SpelEvaluationException: EL1010E: Property or field 'headers' cannot be set on object of type 'org.springframework.messaging.support.GenericMessage' - maybe not public or not writable?

Definition: java -jar http-source-kafka-3.2.1.jar --spring.cloud.stream.bindings.output.destination=output --spring.cloud.stream.function.definition=httpSupplier,spelFunction --spel.function.expression=headers=null --spring.cloud.stream.function.bindings.httpSupplierspelFunction-out-0=output

gauravsingh96 commented 1 year ago

Ok I see spelFunction is not the right function to do this. I am now doing it through headerEnricherFunction but I have to specify each header property and set it to empty string. Is there a way I can set all headers to empty string and not have to specify each separately. The new definition looks like

java -jar http-source-kafka-3.2.1.jar --spring.cloud.stream.bindings.output.destination=output --spring.cloud.stream.function.definition=httpSupplier,headerEnricherFunction --header.enricher.headers=prop1='' \n prop2='' \n prop3='' --header.enricher.overwrite=true --spring.cloud.stream.function.bindings.httpSupplierheaderEnricherFunction-out-0=output

corneil commented 1 year ago

It seems that the best way to solve this is using the HeaderFilter from spring-integration. We will look at adding a header-filter processor that wraps the functionality like header-enricher wraps HeaderEnricher from spring-integration

gauravsingh96 commented 1 year ago

@corneil : I am assuming that this new function headerFilterFunction would be auto-configured to be used in composite function definition in any source application ( as is currently in my definition with headerEnricher). Also, any timeline when this could be released ?

PS - Looking forward to this feature.

corneil commented 1 year ago

We are preparing the release of 4.0.0-M1 soon which also upgrades the stream-apps to Boot 3. If you follow the header-enricher-function and processor code you should be able to create your own processor to remove headers. There are 2 headers which you cannot delete: id and timestamp

gauravsingh96 commented 1 year ago

I am actually trying to avoid creating a separate processor. So I think best would be to wait till you guys configure it in the source applications.

artembilan commented 1 year ago

See spring.cloud.stream.bindings.output.producer.header-mode=none: https://docs.spring.io/spring-cloud-stream/docs/current/reference/html/spring-cloud-stream.html#_producer_properties

corneil commented 1 year ago

Try Maven artifact: org.springframework.cloud.stream.app:header-filter-processor-kafka:4.0.0-SNAPSHOT org.springframework.cloud.stream.app:header-filter-processor-rabbit:4.0.0-SNAPSHOT We should be publishing the docker image soon.

corneil commented 1 year ago

Docker:

onobc commented 1 year ago

Closing via workaround in https://github.com/spring-cloud/stream-applications/issues/442#issuecomment-1497586720