Netflix / suro

Netflix's distributed Data Pipeline
Apache License 2.0
794 stars 170 forks source link

Fix issues that ElasticSearchSink may retry indefinitely on exception #246

Closed allenxwang closed 9 years ago

allenxwang commented 9 years ago

ElasticSearchSink will automatically re-enqueue messages if the batch failed due to an exception. It is discovered that a NullPointerException might be thrown which causes the indefinitely retries on the batch. Also, it will by default sleep 60 seconds before re-enqueuing which causes huge delay during the retry cycles.

The pull request tries to address the following issue:

cloudbees-pull-request-builder commented 9 years ago

NetflixOSS » suro » suro-pull-requests #64 SUCCESS This pull request looks good

cloudbees-pull-request-builder commented 9 years ago

suro-pull-requests #232 SUCCESS This pull request looks good

cloudbees-pull-request-builder commented 9 years ago

NetflixOSS » suro » suro-pull-requests #65 SUCCESS This pull request looks good

stevenzwu commented 9 years ago

LGTM

cloudbees-pull-request-builder commented 9 years ago

suro-pull-requests #233 FAILURE Looks like there's a problem with this pull request