Verify that all output type classes have at most once delivery .

nsoft / jesterj

Document Ingestion Framework for Search Systems

Apache License 2.0

35 stars 33 forks source link

Verify that all output type classes have at most once delivery . #171

Closed nsoft closed 1 year ago

nsoft commented 1 year ago

This should be the default delivery guarantee, obviously minimizing the cases where we fail to send via persistence. Future features will allow enabling "at least once".

nsoft commented 1 year ago

As part of this we will want to add one more status for "attempting to send" that gets set before any output, and warning logs/output for any cases where attempting to send is the latest status (but it does not trigger an FTI re-feed of the document).

nsoft commented 1 year ago

It seems that fetchURL processor only does GET requests, and since mutating get requests are a bad thing, we won't bother to consider it an output, which means only SendToSolrProcessor needs adjustment.