JasonAlt / GridFTP-DSI-for-HPSS

GridFTP module that allows the Globus server to work with HPSS
Other
6 stars 2 forks source link

Query stage status for long running processes #33

Closed JasonAlt closed 5 years ago

JasonAlt commented 5 years ago

When the Transfer services connects and requests to stage files, the DSI will coalesce the requests in order to avoid generating too many async stage tasks within HPSS. The DSI will only issue the request once per bit file ID per GridFTP process.

If the GridFTP process lasts long enough without any successful staging, it is possible that the stage requests have failed or been administratively terminated and the DSI would have no idea; it would just keep using xattrs to determine if the file is available on disk.

The DSI should make use of the async stage status API in HPSS to determine if a request needs to be re submitted.