search

ORNL / DataFed

A Federated Scientific Data Management System
https://ornl.github.io/DataFed/
Other
18 stars 14 forks source link

DataFed is reporting transfer complete before it actually is complete #709

Closed dvstans closed 3 years ago

dvstans commented 3 years ago

When downloading may records (~100), DataFed will report that the transfer has completed before all the transfers are completed. This is likely a bug or missed status check in the Globus transfer polling code in the Core server.

dvstans commented 3 years ago

The cause of this problem is that Globus sends out "success" events periodically on long transfers. DataFed misinterpreted these to mean the entire transfer had succeeded. The only correct way to monitor transfer status is to poll for both task status and task events. A task status of "SUCCEEDED", "FAILED", and "INACTIVE" are definitive; however, there are events that can occur while status is "ACTIVE" that Globus considers transient error, but DataFed considers fatal. For this reason, DataFed must also get the task event list to detect these errors when the task status is "ACTIVE".

dvstans commented 3 years ago

Fixed in ver 1.2