search

apache / camel-k

Apache Camel K is a lightweight integration platform, born on Kubernetes, with serverless superpowers
https://camel.apache.org/camel-k
Apache License 2.0
868 stars 348 forks source link

Incorrect status handling of beersource when it can not connect to its 3rd party web-service #5528

Closed matzew closed 5 months ago

matzew commented 6 months ago

What happened?

The beer source is currently unable to connect to the web-api it is using, and inside the logs we see:

2024-05-21 09:02:24,524 WARN  [org.apa.cam.com.tim.TimerConsumer] (Camel (camel-1) thread #1 - timer://beer) Error processing exchange. Exchange[]. Caused by: [org.apache.camel.http.base.HttpOperationFailedException - HTTP operation failed invoking https://random-data-api.com/api/beer/random_beer with statusCode: 500]: org.apache.camel.http.base.HttpOperationFailedException: HTTP operation failed invoking https://random-data-api.com/api/beer/random_beer with statusCode: 500
    at org.apache.camel.component.http.HttpProducer.populateHttpOperationFailedException(HttpProducer.java:452)
    at org.apache.camel.component.http.HttpProducer.lambda$process$1(HttpProducer.java:275)
    at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:247)
    at org.apache.camel.component.http.HttpProducer.executeMethod(HttpProducer.java:476)
    at org.apache.camel.component.http.HttpProducer.process(HttpProducer.java:246)
    at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:65)
    at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:210)
    at org.apache.camel.processor.errorhandler.NoErrorHandler.process(NoErrorHandler.java:46)
    at org.apache.camel.impl.engine.CamelInternalProcessor.processNonTransacted(CamelInternalProcessor.java:354)
    at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:330)
    at org.apache.camel.processor.Pipeline$PipelineTask.run(Pipeline.java:102)
    at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.doRun(DefaultReactiveExecutor.java:199)
    at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.executeReactiveWork(DefaultReactiveExecutor.java:189)
    at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.tryExecuteReactiveWork(DefaultReactiveExecutor.java:166)
    at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:148)
    at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:59)
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:163)
    at org.apache.camel.impl.engine.CamelInternalProcessor.processNonTransacted(CamelInternalProcessor.java:354)
    at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:330)
    at org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:293)
    at org.apache.camel.component.timer.TimerConsumer$1.doRun(TimerConsumer.java:164)
    at org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:136)
    at java.base/java.util.TimerThread.mainLoop(Timer.java:566)
    at java.base/java.util.TimerThread.run(Timer.java:516)

However, the status is ready:

k get pipes.camel.apache.org 
NAME               PHASE   REPLICAS
beer-source-pipe   Ready   1

That is misleading

Steps to reproduce

  1. install beer source (and see that currently it is unable to connect to its endpoitn, this is not the real reason for the issue being created)
  2. notice it becomes ready, even it is unable to connect to its Web-Api, that it uses.

Relevant log output

No response

Camel K version

2.3.1

squakez commented 5 months ago

This is not really a bug. Let's observe how the Kamelet is designed:

  template:
    from:
      parameters:
        period: '{{period}}'
      steps:
      - to: https://random-data-api.com/api/beer/random_beer/fail
      - to: kamelet:sink
      uri: timer:beer

Every given time, the kamelet emit an exchange reading to the API. The fact the API is failing is not a hard failure by the component used. You can have a deeper look at the Camel Http component documentation. As you can see, it is in fact reported as a "Warning". And, to a certain extent it makes sense, as the http component can fail for any networking reason. If you want to make it a stricter one, you probably need to develop the logic by checking the http codes and working with circuit-breaker eips or leveraging Pipes error handler.