search

jash-kothari-forks / libtorrent

Automatically exported from code.google.com/p/libtorrent
Other
0 stars 0 forks source link

zombie sparse flag remains set on fully downloaded files #419

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
A torrent containing a Matroska video file downloads completely with no hashing 
errors, yet a re-check produces an I/O error. The sparse flag seems to be 
involved in some way.

What steps will reproduce the problem?

1.Using qBittorrent 3.0.8 download the following to 100% (6.8GB):

magnet:?xt=urn:btih:411a7a164505636ab1a8276395b375a3a30bff32&dn=TPB.AFK.2013.108
0p.h264-SimonKlose&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2
Ftracker.publicbt.com%3A80&tr=udp%3A%2F%2Ftracker.istole.it%3A6969&tr=udp%3A%2F%
2Ftracker.ccc.de%3A80

The film is under a CC-BY-NC-ND license. Sharing via torrent is encouraged by 
the author.

2. After downloading to 100%, right click -> force re-check on the torrent

3. At some point re-checking stops with the following error message: "An I/O 
error occured for torrent TPB.AFK.2013.1080p.h264-SimonKlose. Reason: 
TPB.AFK.2013.1080p.h264-SimonKlose file 
(C:</path/to/>TPB.AFK.2013.1080p.h264-SimonKlose.mkv) error: The file or 
directory is corrupt."

What is the expected output? What do you see instead?

Expected output is a working mkv file, since it downloads to 100% with no 
hashing errors at first. Instead playing back 
TPB.AFK.2013.1080p.h264-SimonKlose.mkv in MPC-HC from the CCCP codec pack 
(http://www.cccp-project.net) resulted in video and audio freezing at a certain 
point. This is how i first noticed the problem, which lead me to do the 
re-check that produces the above mentioned error.

Re-checking the same file in utorrent 2.2.1 also fails with "error: the file or 
directory is corrupted and unreadable".

What version of the product are you using? On what operating system?

qBittorrent 3.0.8 which uses libtorrent 0.16.7.

OS: Windows 7 SP 1 64bit fully patched.

I was able to reproduce the error on another machine, OS Win 7 SP 1 32bit fully 
patched, by downloading the torrent again.

Please provide any additional information below.

The reason i'm calling this the zombie sparse flag 
(https://code.google.com/p/libtorrent/issues/detail?id=143) is that right click 
-> properties, details tab shows "Attributes AP". To my knowledge P stands for 
Sparse 
(http://serverfault.com/questions/34692/windows-file-and-folder-attributes-a-or-
c/34711#34711). Yet checking the file with "fsutil sparse queryflag" reveals 
that "This file is NOT set as sparse".

Setting the file as sparse using "fsutil sparse setflag", it becomes playable 
in MPC-HC without freezing. Re-checking the file in utorrent (to avoid 
qBittorrent re-setting the sparse flag) afterwards shows it to be 99.7% 
complete.

I'll be happy to provide any additional info if needed.

Original issue reported on code.google.com by emptyeig...@gmail.com on 12 Feb 2013 at 7:07

GoogleCodeExporter commented 9 years ago 
libtorrent is supposed to clear the sparse file flag for any file as soon as it 
completes. It does this by doing this:

      DWORD temp;
      FILE_SET_SPARSE_BUFFER b;
      b.SetSparse = FALSE;
      ::DeviceIoControl(m_file_handle, FSCTL_SET_SPARSE, &b, sizeof(b)
         , 0, 0, &temp, 0);

Now, there could be a bug where this fails to be executed. Would you think 
that's the case?

The whole thing with re-checking to 99.7% seems odd as well, is that related to 
the sparse flag? Does that change when you set or clear the sparse flag?

Is the main purpose of having this flag cleared media players performing better?

Original comment by arvid.no...@gmail.com on 13 Feb 2013 at 3:08

GoogleCodeExporter commented 9 years ago 
I've checked in a patch that removes the concept of finalizing a file (i.e. 
clear the sparse bit) and made that logic more conservative.

Whenever a file open for write mode is closed, and the sparse bit has been set 
on it. The file is checked to see if it is in fact sparse. If it isn't, the 
sparse flag is cleared. While fixing this, I also found some potential issues 
with not passing in OVERLAPPED pointers to some disk operations, when they 
would need them. This was also fixed.

The commit is [8015] in RC_0_16.

My testing suggests that it works as intended, I never observed the zombie 
sparse flag though. Please test this if  convenient.

Original comment by arvid.no...@gmail.com on 16 Feb 2013 at 8:14

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
confirmed here as well on windows 7 x64 with qbittorrent 3.1.0 ALPHA funny 
thing is i ended up RMAing two hard drives before coming here -.- 

file would play and then abruptly stop was able to repeat the process anytime 
that file was downed loaded anyware on c: how ever if I moved the file to d: 
the issue did not occur so I assumed the drive was as fault got the new drive 
in same issue setting the fsutil sparse flag resolved the issue perhaps there 
is some conflict between the fs handles on the OS drive
anouther note is that the issue only seems to occur with file sizes >6.5GB

Original comment by rjc862...@gmail.com on 25 Feb 2013 at 2:45

GoogleCodeExporter commented 9 years ago 
work around
1. fsutil sparse setlflag filenameinquestion
2. Open QbitTorrent and force a recheck wait for it to redownload the missing 
bits
3. close Qbittorrent and do fsutil sparse setlflag filenameinquestion again
4. copy the file elseware and enjoy 
5. if you accidently open Qbittorent and its still has that torrent loaded you 
will need to repeat this process untill you either remove the torrent or copy 
the file elseware

Original comment by rjc862...@gmail.com on 25 Feb 2013 at 3:11

GoogleCodeExporter commented 9 years ago 
For reference, I'm posting the link to the knowledge base article here as well. 
This suggests that the sparseness of the file itself causes the problem.

  http://support.microsoft.com/kb/2708811?wa=wsignin1.0

Original comment by arvid.no...@gmail.com on 25 Feb 2013 at 5:08

GoogleCodeExporter commented 9 years ago 
> http://support.microsoft.com/kb/2708811?wa=wsignin1.0

This may have something to do with 
https://code.google.com/p/libtorrent/issues/detail?id=406

The article also states that this happens when two threads write the same file 
part at the same time, which is one of my possible explanations for Bug #406.

Original comment by Daymansm...@gmail.com on 25 Feb 2013 at 5:24

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
after checking my system 
KB2708811(http://support.microsoft.com/kb/2708811?wa=wsignin1.0) was already 
installed 
would not explain setting the flag causing the problem to resolve anyway 
This fix is delivered via windows update anyway so anyone with a up2date 
machine should already have this hot fix

Original comment by rjc862...@gmail.com on 25 Feb 2013 at 10:55