Corrupt files when copying

[jdenman03]

qBittorrent & operating system versions

MAC OS 12.3.1 qBittorrent 4.4.2 x64

What is the problem?

After downloading a file, there is a size discrepancy upon finishing. I have posted screenshots of the issue. What's odd is that the file will play perfectly fine on the same HDD it is downloaded to, but once I copy the file to an external drive or to my NAS the file becomes corrupt.

HOWEVER, if I copy the file and paste it to the same HDD, the file sizes get corrected and then I can copy the file to my external HDD or to my NAS without any problems.

I initially thought that it was a MAC OS problem, but I upgraded to 12.3.1 and the problem is still there.

Steps to reproduce

No response

Additional context

No response

Log(s) & preferences file(s)

No response

[jdenman03]

4.4.4 came out today. I will try it tonight and report back.

Get Outlook for Androidhttps://aka.ms/AAb9ysg

---

From: dldingfls @.> Sent: Wednesday, August 24, 2022 5:17:20 PM To: qbittorrent/qBittorrent @.> Cc: jdenman03 @.>; Mention @.> Subject: Re: [qbittorrent/qBittorrent] Corrupt files when copying (Issue #16843)

Downgrading to 4.3.9 worked perfectly for me too. I lost a lot of hair using version 4.4.3, with glitchy files that pixelated and with image freeze and sometimes total file shut-down. Totally sorted now!

— Reply to this email directly, view it on GitHubhttps://github.com/qbittorrent/qBittorrent/issues/16843#issuecomment-1226370071, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFUXXY3BUYDXLZC3FZN3XJ3V22GOBANCNFSM5S5HFH5A. You are receiving this because you were mentioned.Message ID: @.***>

[ErikEklund]

I have been experiencing the same issue - when I copy the file to my NAS, it's corrupted. I thought the download itself was corrupted, googling it brought me to this issue, and can confirm it works when played from the download folder on my local disk. I tried to install 4.4.4 and re-download a file - the same problem still exists in 4.4.4.

Will also try downgrading, thanks for the tip.

[ghost]

4.4.4 does not have a fix. However you may use 4.4.4 RC_1_2 which should not have this issue.

[arvidn]

Does the copied file end up corrupt when using the internal "move storage" function? Or is this problem only exhibited when copying the file using an external process? If so, it could be a matter of synchronisation between libtorrent flushing/syncing the file and starting the copy.

[loekf]

4.4.4 does not have a fix. However you may use 4.4.4 RC_1_2 which should not have this issue.

Why would 4.4.4.RC_1_2 work ? I tried this one, but I'm getting "couldn't write to disk" error. Looks like this version doesn't request permission to access disk & folders in macOS Monterrey.

So back to 4.3.9

[loekf]

Does the copied file end up corrupt when using the internal "move storage" function? Or is this problem only exhibited when copying the file using an external process? If so, it could be a matter of synchronisation between libtorrent flushing/syncing the file and starting the copy.

From what I've seen (including myself) the issue occurs when you copy a downloaded file from an internal disk or externally connected disk via USB to a network drive (or newly attached/mounted disk ?). So a different disk than the originally downloaded file was on.

Enabling checking after downloading doesn't help btw.

[ghost]

Why would 4.4.4.RC_1_2 work

because it uses the same disk I/O as 4.3.9

[jdenman03]

Why would 4.4.4.RC_1_2 work

because it uses the same disk I/O as 4.3.9

I'll try RC tomorrow morning and report back.

[ghost]

or you may just download 4.4.5 which uses RC_1_2 by default now.

[loekf]

or you may just download 4.4.5 which uses RC_1_2 by default now.

Briefly tried it, not a solution, because I see "couldn't write to disk" errors. Was also with 4.4.4 RC1_2.

RC2.0 or RC1.2 doesn't matter.

I suspect qbittorrent doesn't request write access to removable drives in my case, though the privacy settings look OK.

[ghost]

I believe “couldn’t write to disk error” was not mentioned anywhere by OP and is diverting the topic at hand. It’s most likely a different issue and you should file a new issue ticket with detailed info such as logs, hardware details, configs.

[loekf]

Looks like 4.4.x and 4.3.9 have seperate files to store settings, so 4.4.5 was trying to access an old external disk, which I replaced.

I tried 4.4.5. Copied one file to my NAS. Looks OK, but have to see how it behaves for a longer period.

[Steve-Agnew-NZ]

Confirming that upgrading to 4.4.5 has fixed the problem I was having with files downloading OK but then being corrupted by Finder as they were being copied.

[Guille-M]

Copying issue (to external disks) when having Qbit open has been fixed with update 4.4.5, MacOS 12.5.1, no confirmed issues after copying and opening files.

[ghost]

Copying issue (to external disks) when having Qbit open has been fixed with update 4.4.5, MacOS 12.5.1, no confirmed issues after copying and opening files.

Nothing was "fixed". Just the default build was changed to libtorrent 1.2.17. The problem should still exist if you try the RC_2_0 build.

[superfreeman1989]

I'm having the same issue with corrupted files. They all happen at ~80% progress with the video files. And I downloaded directly to external hard drive without copying. Only discovered the issue after downloading 200gb of stuff. Kinda pissed

[jdenman03]

Have you upgraded to 4.4.5?

[danieljgmaclean]

Just sharing my experience for the record.

I started observing this same issue with video files that were downloaded both with qBittorent and Transmission. I tried different drives but they all had issues. The only things the files had in common with the problem was that they were downloaded using torrent and they were being moved to another drive after download.

I discovered that using the Finder to copy files nearly always corrupted the files. While using the terminal and the mv command, the files always copied 100% perfectly i.e. no stuttering in playback.

To be clear, no other files I download via the browser or other mechanisms resulted in corrupted files. For this reason, I ruled out APFS and macOS in general to be the issue, whether that's a good enough reason, who knows...

All my drives (internal and external) are formatted with APFS. I'm using macOS Monterey 12.3.1.

To fix this problem, I ended up writing a script to use the mv command to shift the files after download rather than doing it manually via the finder. This avoids the issue and the files copy fine and should mean I can use any version of qBittorent.

I've just updated qBittorrent to 4.4.5 and can see it's using Libtorrent 1.2.17.0.

If I see a repeat of the issue above, i'll post an update here.

[pifeifei]

+1

Same question.

CPU: i3-3220 OS: TrueNas scale + docker

qBittorrent v4.4.5 Web UI (64-bit)

Qt: 6.3.1 Libtorrent: 2.0.7.0 Boost: 1.80.0 OpenSSL: 3.0.5 zlib: 1.2.12

[strpyw]

I went to qBittorrent to update my Mac OS 64 bit to v4.4.5 Did one download of 20G and copied the video file to one external drive. Delete the torrent and reload it to the ext drive to check if there is corruption. None, so far so good

[Oh4sh0]

Adding mine. Had qBittorrent 4.4.3 on macOS 12.4.

qBittorent would write to a local temp location and copy to a Synology NAS via SMB upon completion.

Upon completion, video files would have some sort of corruption where a few second segments would be missing throughout the videos.

Simply changing the download path to keep the files local seems to have solved the problem. I am upgrading to 4.4.5, and will revert my config to have qB copy direct upon completion and confirm whether the problem is resolved.

[mbbananas]

Has this worked for everyone? I've tried everything and this still just mysteriously stopped working earlier this year for me. The upgrade to qbittorrent 4.4.5 did not help. There is no work-around for me, the md5 hashes don't match most of the time and I get dropped frames and bleeding. Cp -r and rsync -r do not work. I've tried a lot of 3rd party file transfers like forklift as well to no avail. I even tried utorrent for a second instead of qbittorrent and got the same issue. I have several different externals and they all seem to be working fine. It's mostly on movie transfers and sometimes if I do it 5 different times, it will eventually work, but most of the time I get the dropped frames.

[thuglifem]

I dont have any problems with 4.4.5, as far as i remember I was on monterey 12.4 or 12.5 when it came out. Now I'm on ventura 13.0.1(mba m1). Works fine for me.

[arvidn]

@mbbananas you might want to check your RAM

[rainple-liu]

This problem should be caused by Apple. MacOS has been benchmark cheating in writes by not doing proper fsync.

[mbbananas]

Thanks! After months of trying different things, it seems that it is most likely the use of my USB hub. It was working before so it could be a combination of Apple and a USB hub. If I don't have too many externals connected to the same hub or I connect an external directly, everything seems to copy over fine.

[Stokestack]

Do we have a verdict on whether this is fixed in the current version, 4.5.2?

[loekf]

Yes, as long as you use the version with the older libtorrent.

[Stokestack]

Thanks. Was that version rolled back in the installer? Or is libtorrent installed independently? I don't remember installing it or where it resides.

[loekf]

There are two versions for each build. Just make sure you DONT download the version using lt20, because that one uses libtorrent 2.0 with a new file caching algo, which corrupts files on Macos.

[Stokestack]

Thanks. Why is that one still up, then?

A quick search didn't reveal any issues raised on this in the libtorrent repo, so I posted one.

[arvidn]

the problem people are talking about is still when copying the downloaded file, right? Not when playing/reading the file from the disk it was originally downloaded to. Is that right?

If so, I would be interested to know if it makes a difference to first stop the torrent and then copy the file, or maybe even shut down qbittorrent and then copy the file.

Also, just to be clear, the copying does not happen by qbittorrent itself, right? (e.g. move-download-location) but it happens via some other copying tool, like drag and drop in finder or cp or rsync.

[Stokestack]

That's right. The files seem to be fine when they reside where QBittorrent deposited them.

It doesn't make a difference if you stop the torrent. I've copied the files using Finder without qBittorrent even running, and they get corrupted. I tried two different drives (thumbdrive and SSD) over USB, and even copied the corrupted file back to the Mac... where it was still bad (showing that playback from the external drive wasn't the problem).

[MrGarak]

I have/had this exact issue as well with videos. I went crazy for months trying to sort it out, when it comes down to it, qBittorrent was causing this to happen for me. I would a video download to a folder locally on my mac mini, then copy to an external drive. Every single thing I copied would become corrupt and show these issues.

Switched to Transmission and have not had this issue once.

[michaelvolz]

I am using 4.5.5, and it seems I have now the same problem. Multiple files corrupted after moving to another drive. Is this a regression? I am the only one?

I moved the files back and made a recheck and indeed the files were incomplete, and some chunks needed to be downloaded again.

I enabled re-check after move and will see if it changes anything.

[thuglifem]

Which version of 4.5.5 are you using? The standard or the lt20? As far as I'm aware people don't have problems with the standard version, just with the lt20 one. p.s. I don't have any issues with the standard 4.5.5, I can confirm that because it's not brand new version, i've been using it for a while.

[ram4life]

I've found that once a torrent finishes, right-clicking on it and doing a "force recheck" two or three times allows the client to check all the parts and I don't get the playback issues anymore. maybe this'll help?

[Stokestack]

Has the lib still not been fixed?

[michaelvolz]

fyi: My Issue is resolved, it was NOT QBittorrent.

After the move, the path was too long and produced a false file corrupted error from the Installer.

[datcactus]

I can confirm that even on a non-Mac system this is very easy to replicate with larger torrents (>10GiB in size). Downloading to an SSD and then moving (via NFS) to a ZFS array regularly causes 99.9% recheck failures which require moving back to the download directory to re-fetch the .1% corruption, move back to the array, and hope for the best. This is present on 4.6.2 so there has been no fix/improvement in this behavior over the 4.5.5 version I was on previously.

[rebusx]

@arvidn

the problem people are talking about is still when copying the downloaded file, right? Not when playing/reading the file from the disk it was originally downloaded to. Is that right? Also, just to be clear, the copying does not happen by qbittorrent itself, right? (e.g. move-download-location) but it happens via some other copying tool, like drag and drop in finder or cp or rsync.

If you still need this info: in my case, i moved the files via "change download location" in qbittorrent like you say, during downloading. The torrent was huge, 200 GB or so, and individual files would be 1-3 GB. That is 100% sure. Also relevant, another download like that was happening at the same time, also big one, and at least one of them gave me hash errors. I did a recheck on that one. Ended up with an interesting corruption where a video file is mixed up with another file from that other download, i.e. a segment of another file had replaced a segment that is supposed to be. I'm certain this is not a filesystem or hardware issue, because none of the other large or media files on that disk got this problem, only the ones donwloaded with qbittorrent; and also, there was no any glitches in audio, which are guaranteed to be when there is an FS problem. In other words, a wrong segment was perfect by its data (video and audio tracks without issues), except it was from a wrong media file which of course gave structure errors in that file. So it looks exactly like "wrong piece". Also, this was on NTFS in Windows, an external HDD connected via USB. But i can't tell you any versions :( this download happened in 2021 and i don't use qbittorrent for some time now

[eisenbruch]

I had this issue on my mac for the last few weeks it seems. Was driving me nuts.

I switched to using v4.6.3 NOT THE LT20 VERSION and it seems to have resolved the issue so far.

It seems insane that this bug still exists

[wpwentzell]

I can confirm that even on a non-Mac system this is very easy to replicate with larger torrents (>10GiB in size). Downloading to an SSD and then moving (via NFS) to a ZFS array regularly causes 99.9% recheck failures which require moving back to the download directory to re-fetch the .1% corruption, move back to the array, and hope for the best. This is present on 4.6.2 so there has been no fix/improvement in this behavior over the 4.5.5 version I was on previously.

+1. I stumbled onto this issue today after I had been backing up downloaded files to a local SMB share on another machine. Recently, I installed Plex on that machine and used those backup drives as media drives for Plex, and noticed that files larger than 10GB or so were glitchy/corrupted and would not play back correctly. After running a bit-by-bit comparison check using FreeFileSync, I noticed it was only the larger torrent files that had been corrupted. This is really frustrating, because now I cannot trust my files to get backed up without corruption. Not sure where the problem lies, but it's only happening to files that were downloaded using qbittorrent and then copied over to the SMB share.

I'm on Windows 11 and was using qbittorent 4.4.3. I have since updated to 4.6.3 to see if it alleviates the issue.

[vincejv]

+1 i can confirm this issue as well not mac, but on linux BTRFS ssd to BTRFS hdd moving files through incomplete/complete download feature on torrent client, it starts to seed after the files were moved from incomplete folder to complete then detects then midway it detects a "corruption", then redownloads the file again, probably due to the new broken disk io on v2, i'm reverting to Libtorrent v1.

Weird thing is it's consistent, when it's corrupted (doesn't happen all the time), only 1 piece (or the the last piece) gets corrupted and redownloaded, and the rest of the file seems to pass the hash check, so I don't see it being a hardware or ram corruption issue.

I have noticed this issue of Libtorrent 2 only, not on Libtorrent 1, using the latest Linux Kernel LTS v6.6.23. As previous users mentioned, mostly occurs when downloading very large files >20gb

@glassez Please consider this issue when QBT 5.0 is released, don't think we should rush on sticking with Libtorrent v2, i highly suggest to please keep both the libtorrent v1 and v2 builds.

[xavier2k6]

@vincejv @wpwentzell @eisenbruch How does qBittorrent 4.6.4 which uses Libtorrent 2.0.11.0 behave?

[vincejv]

if QBT 4.6.4 uses the unreleased 2.0.11.0, then the behavior is the same. I'm using QBT 4.6.4.

[glassez]

@glassez Please consider this issue when QBT 5.0 is released, don't think we should rush on sticking with Libtorrent v2, i highly suggest to please keep both the libtorrent v1 and v2 builds.

We returned to using libtorrent-1.2 by default a long time ago, when it became clear that the libtorrent-2.0 I/O subsystem was causing serious problems in some (fairly common) use cases.

[Stokestack]

@glassez Cool, thanks. So it's safe to let it auto-update?

[glassez]

@glassez Cool, thanks. So it's safe to let it auto-update?

If you are referring to the built-in updater, then it just downloads the default installer package for your OS, which is currently based on libtorrent-1.2.