ORNL / DataFed

A Federated Scientific Data Management System
https://ornl.github.io/DataFed/
Other
18 stars 13 forks source link

Add retries as option when working with data transfers #919

Open JoshuaSBrown opened 9 months ago

JoshuaSBrown commented 9 months ago

Description

In the face of network faults, DataFed is canceling tasks. This needs to be controlled with an option. Instead, the default Globus behavior should be used, if a user specifically wants to cancel a transfer after a determined number of faults that is an option they can specify. This is particularly important when first testing a repository and attempting to detect network errors, but it is also useful to cancel tasks early in cases where there is high network traffic and an alternative transfer request to a different Globus endpoint might be preferable.

Tasks

Work time

Expected time: 24 hours

JoshuaSBrown commented 9 months ago

Still need to address the faults logic in the GlobusAPI.cpp file and update the documentation. Also need to update the web server to support a retries parameter.

JoshuaSBrown commented 8 months ago

@par-hermes format