Slow loading files fail (over 1 hour)

What steps will reproduce the problem?
1. Load a large file, or load over a slow network
2. Watch the the tempfile table in the database
3. Watch the actual file on the storage node

What is the expected output? What do you see instead?
I expect my slow loading file to be loaded.

After an hour the tempfile entry disappears and the file on the storage node is 
deleted. My load thread completes but MogileFS never responds to create_close 
command (timed out after 10 mins for me) and the file is not present.

I'm no good with perl but I think the problem seems to be in the Delete worker:
    # dig up some temporary files to purge
    my $sto = Mgd::get_store();
    my $too_old = int($ENV{T_TEMPFILE_TOO_OLD} || 3600);
    my $tempfiles = $sto->old_tempfiles($too_old);
    return 0 unless $tempfiles && @$tempfiles;

Fixes
 - it would be best if the worker could check to see if the file is still been written to before culling it so I don't need to raise the value.
 - If I have to raise the value can we set it in the config file rather than an environmental variable?

Thanks!

Original issue reported on code.google.com by simon...@gmail.com on 20 Mar 2013 at 9:33

glyustb / mogilefs

Slow loading files fail (over 1 hour) #77