Is there any option to retrieve error result from underlying output component that is wrapper inside Retry output component?

[pradeeppattamatta]

In the below config, output_pubsub is gcp_pubsub component. When there is an exception returned from that component, I see it in the error log.

output:
  label: "output_pubsub_fallback"
  fallback:
  - retry:
      max_retries: 3
      backoff:
        initial_interval: 5s
        max_interval: 50s
        max_elapsed_time: 10m
      output:
        resource: output_pubsub
  - broker:
      pattern: fan_out_sequential
      outputs:
        - resource: log_and_metric
        - resource: drop_to_dlq_kafka

error_log

{
    "@service": "benthos",
    "label": "output_pubsub",
    "level": "error",
    "msg": "Failed to send message to gcp_pubsub: failed to get topic: test-topic-2: failed to validate topic 'test-topic-2': rpc error: code = Unauthenticated desc = transport: per-RPC creds failed due to error: oauth2: cannot fetch token: 400 Bad Request\nResponse: {\n  \"error\": \"invalid_grant\",\n  \"error_description\": \"Bad Request\"\n}",
    "path": "root.output_resources"
}

But when I want to use that in the log_and_metric component to extract the error type and add metric and log, the fallback_error has the field "fallback_error":"message failed to reach a target destination" and not the actual error message.

If I remove the retry component and directly configure output under fallback, the fallback_error correctly returns the error from pubsub.

output:
  label: "output_pubsub_fallback"
  fallback:

  - resource: output_pubsub
  - broker:
      pattern: fan_out_sequential
      outputs:
        - resource: log_and_metric
        - resource: drop_to_dlq_kafka

fallback_error field as expected - "fallback_error":"failed to get topic: test-topic-2: failed to validate topic 'test-topic-2': rpc error: code = Unauthenticated desc = transport: per-RPC creds failed due to error: oauth2: cannot fetch token: 400 Bad Request\nResponse: {\n \"error\": \"invalid_grant\",\n \"error_description\": \"Bad Request\"\n

How to propagate the error from output components under retry to outside components?

[pradeeppattamatta]

Also, is there any way to retrieve the message id from pubsub publisher response ?

[pradeeppattamatta]

Hi @mihaitodor - Can you pls take a look and let me know your thoughts!