michael-conway
commented
7 years ago Did you specify a callback listener? The way it works is that, if a callback listener is specified, the exception goes to the callback listener, rather than gets rethrown. This was originally done for the old desktop transfer client, and a few other use cases, where the model was to tolerate partial failures, letting the callback listener decide what to do (e.g. a model to set a tolerance level so that the user could later retransmit only failed operations).
So in the code below, I could reconsider that and have a 'fail fast' setting in Jargon properties that will go ahead and just rethrow up in the put operation.
The way this works now is that jargon is deferring to the callback listener to decide how to handle the exception. If you did NOT specify a callback listener, let me know, because that would be a 'bug' not a 'feature'!
I'm happy again to add a 'fail fast' configuration option and set that as the default. I suspect I'll be re-approaching desktop transfer tools and I can just turn off fail fast in that scenario.
private void processExceptionDuringPutOperation(
final File sourceFile,
final IRODSFile targetIrodsFile,
final TransferStatusCallbackListener transferStatusCallbackListener,
final TransferControlBlock transferControlBlock,
final JargonException je) throws JargonException {
log.error("exception in transfer", je);
transferControlBlock.reportErrorInTransfer();
if (transferStatusCallbackListener != null) {
log.warn("exception will be passed back to existing callback listener");
TransferStatus status = TransferStatus.instanceForException(
TransferType.PUT, sourceFile.getAbsolutePath(),
targetIrodsFile.getAbsolutePath(), "", sourceFile.length(),
sourceFile.length(),
transferControlBlock.getTotalFilesTransferredSoFar(),
transferControlBlock.getTotalFilesSkippedSoFar(),
transferControlBlock.getTotalFilesToTransfer(), je,
getIRODSAccount().getHost(), getIRODSAccount().getZone());
transferStatusCallbackListener.overallStatusCallback(status);
} else {
log.warn("exception will be re-thrown, as there is no status callback listener");
throw je;
}
In a related ticket (#132), we noticed log messages like those:
However, Jargon does not propagate errors up from the multithreaded parallel file transfer code to the main DataTransferOperations.putOperation method. Although we see these errors in the log file, our call of the putOperation returns as if it was successful.
I think it is important to fail with exceptions whenever a change was not persisted to iRODS (same is true for the icommands of course). Please extend exception handling to cover multithreaded cases in Jargon.