Question: Is it possible to run a step (or await on a job) that is not run inside the "acidic" transaction.
Use case: Let's say we want to fetch a collection of items from an API, process each item, then persist the processed items. The collection feature in AcidicJob can do this, but let's say an additional requirement is that we don't want to persist the collection to a database record.
Let's also assume that processing is expensive, so while we don't mind fetching from the API on retry, we don't want to reprocess items on retry.
One way we could handle this is with an idempotent job that: 1) fetches the collection; 2) checks the db to see which processed items exist; 3) only processes the unprocessed items from the collection (persisting immediately as each item is processed). If there is a failure during the processing iteration, we can retry the job, and it should start from where we left off, but that only works if we are outside of the "acidic" transaction (because otherwise all the persisted processed items are rolled back in the event of an error). Hope this makes sense.
Maybe there is another way of thinking about it within the AcidicJob frame of mind that I'm not seeing?
Question: Is it possible to run a step (or await on a job) that is not run inside the "acidic" transaction.
Use case: Let's say we want to fetch a collection of items from an API, process each item, then persist the processed items. The collection feature in AcidicJob can do this, but let's say an additional requirement is that we don't want to persist the collection to a database record.
Let's also assume that processing is expensive, so while we don't mind fetching from the API on retry, we don't want to reprocess items on retry.
One way we could handle this is with an idempotent job that: 1) fetches the collection; 2) checks the db to see which processed items exist; 3) only processes the unprocessed items from the collection (persisting immediately as each item is processed). If there is a failure during the processing iteration, we can retry the job, and it should start from where we left off, but that only works if we are outside of the "acidic" transaction (because otherwise all the persisted processed items are rolled back in the event of an error). Hope this makes sense.
Maybe there is another way of thinking about it within the AcidicJob frame of mind that I'm not seeing?