search

vkuznet / transfer2go

Distributed, loosely couple agent-based transferring system
MIT License
8 stars 2 forks source link

Increase delay for 204 error #59

Closed rishiloyola closed 6 years ago

rishiloyola commented 6 years ago

Here I am removing Error() because our system is breaking whenever the err is nil.

rishiloyola commented 6 years ago

Test result for these new changes - Destination screen shot 2017-10-19 at 6 09 22 pm Main agent screen shot 2017-10-19 at 6 09 28 pm

vkuznet commented 6 years ago

Rishi, your general observation was correct that we should stop requests from indefinite loop when file is not there and in this PR you partially addressed this. I made proper changes here: 2034adf..f90f98c so I'll discard this PR. The Stager make time to stage file, i.e. it is not immediate action link in case of making symlink. In other circumstances Stager will take time to stage file from a tape system. Therefore we need to put request on hold (as you did in this PR, see your change https://github.com/vkuznet/transfer2go/pull/59/files#diff-a45e2c5e3f1f075ead25a80f5d0d5052R277). But you shouldn't modify 204 from processing to notfound, since in general 204 means no content which is much broader view. Either file does not exists or it may be in a process of being staged from tape which means it does not exists YET. I also made transfer delay configurable in my commit, such that sites can adjust their settings according to their storage latencies.