Torrent stall: "There is not enough space on the disk"

Mike-EE commented 4 years ago

Please provide the following information

qBittorrent version and Operating System

qBt 4.1.9.1 on Windows 7-64 Home SP1

What is the problem

Starting v4.1.9, downloading torrents to QNAP NAS (QTS 4.4.1.1117) results in intermittent stalls with the error message "There is not enough space on the disk".

18TB of free space on NAS
Does not happen to all torrents, perhaps 60%
The affected torrents will restart after about 5 minutes 90% of the time
Some need to be rechecked by "Force recheck" - they will restart most of the time
No apparent pattern - happens regardless of size or number of files, or source - could occur with torrents from any tracker
Does not happen to torrent files saved to the resident (internal) HD
Moving from HD to NAS via "Set location" behaves as expected
Did not occur prior to v4.1.9

What is the expected behavior

Return to 4.1.7 behavior

Steps to reproduce

1 - Open torrent and select QNAP NAS network shared folder as the save target. 2 - Observe behavior and repeat as need until issue is reproduced.

Extra info(if any)

What additional information is needed from me to help diagnose and correct?

Mike-EE commented 4 years ago

Can the developers please review this issue and let me know what additional information is needed?

FranciscoPombal commented 4 years ago

Have you tried reproducing in 4.2.1 or the latest master (in case you are able to compile from source)?

In case you are able to reproduce the issue, I'd say more details about your storage configuration would be helpful, (RAID type, filesystems being used, what is internal or external, in which disk qBittorrent is installed), etc.

Try to provide as much details as possible. Your setup is pretty specific and not many people will have something equivalent they can try to reproduce the problem on.

Mike-EE commented 4 years ago

Have you tried reproducing in 4.2.1 or the latest master (in case you are able to compile from source)?

Yes - all versions after 4.1.7 exhibit the issue, though I haven't tried to compile any of the development builds.

In case you are able to reproduce the issue, I'd say more details about your storage configuration would be helpful, (RAID type, filesystems being used, what is internal or external, in which disk qBittorrent is installed), etc.

Try to provide as much details as possible. Your setup is pretty specific and not many people will have something equivalent they can try to reproduce the problem on.

qBt is installed on a Win7-64 SP1 system. The save-target system is a QNAP TS-451A running the latest Ubuntu Linux-based QTS firmware (4.4.1.1117). The NAS contains three Seagate 12TB Iron Wolfs running in JBOD mode and formatted in ext4.

Are there specific logs or WireShark captures that would help?

FranciscoPombal commented 4 years ago

Are there specific logs or WireShark captures that would help?

Not sure tbh. But I don't think that is the best course of action, read on.

Yes - all versions after 4.1.7 exhibit the issue, though I haven't tried to compile any of the development builds.

If you can, try compiling 4.1.7 with the same Qt, libtorrent, etc versions as the 4.1.9 release was compiled. If the issue disappears, we can most likely confirm that the issue lies with the dependencies.

If not, use git bisect between the 4.1.7 and 4.1.9 tags to find the commit that introduced the regression.

FranciscoPombal commented 3 years ago

@Mike-EE

Any info on this? Did you ever get round to bisecting this? If not, can you reproduce it in the most recent version? Similar issue: https://github.com/qbittorrent/qBittorrent/issues/13680

Mike-EE commented 3 years ago

@FranciscoPombal -

Thanks for following up on this.

This still occurs, even with the latest versions, any time that I try to download a torrent from the qbt Windows client to my Linux-based NAS. I've read several threads alluding to similar, or perhaps identical (e.g. the issue you posted), issues with the same behaviour pattern: start download to storage system with more than enough room, the download halts and an error message is displayed by qBt, wait 5 minutes, typically restarts.

I have not had the chance to diagnose this - I've been using the work-around of saving to a local disk, then move it to my NAS. I've been hoping that someone else with the time could provide dev team with data that they need to diagnose this as I lost my build system and haven't had the time to stand-up my build systems again; I'm just a 'punter' and don't need a build system on a daily basis.

FranciscoPombal commented 3 years ago

@Mike-EE

No problem. I've seen other issues reporting problems related to the usage of network shares or other types of more exotic storage other than simple local disks; though I can't find them now, and don't know what's the status of those issues in recent versions. But I would say that your issue is a mixture of those issues plus https://github.com/qbittorrent/qBittorrent/issues/13680. Let's help someone else can come along and help.

FranciscoPombal commented 3 years ago

@Mike-EE

This may be fixed with https://github.com/qbittorrent/qBittorrent/pull/14825. You can download test build for Windows that includes the fix here (under "Artifacts"): https://github.com/qbittorrent/qBittorrent/actions/runs/769568755.

Mike-EE commented 3 years ago

@FranciscoPombal

Not fixed and the behavior is actually worse with the test build:

Behavior with all production builds since my original report: approximately 30% of torrents (varying sizes and download speeds, no obvious pattern) will pause downloading, print the File error alert. Torrent: XXXXXXX error: There is not enough space on the disk to the log, will restart downloading after 5 minutes and will successfully complete the download.
Behavior with this build: approximately 30% of torrents (varying sizes and download speeds, no obvious pattern) will stop downloading, print the File error alert. Torrent: XXXXXXX error: There is not enough space on the disk to the log, will change the torrent color to red on the GUI and show an exclamation point, will not restart the downloading and will require a forced recheck to successfully complete the download.

Please let me know if I can provide additional data.

FranciscoPombal commented 3 years ago

@glassez ping, feedback about https://github.com/qbittorrent/qBittorrent/pull/14825.

glassez commented 3 years ago

Please let me know if I can provide additional data.

Screenshots of how does it really looks in both cases are welcome. Also, please explain whether you really have insufficient disk space so your torrents often fall into the situation described above?

Mike-EE commented 3 years ago

Please let me know if I can provide additional data.

Screenshots of how does it really looks in both cases are welcome. Also, please explain whether you really have insufficient disk space so your torrents often fall into the situation described above?

I can provide a screenshot tonight but I can say now that I currently have 6.5TB of free space on the QNAP NAS target (TS-451+ / QTS v4.5.2.1566) with the qBt test build installed on an i7 / Win10-64 Home client PC.

glassez commented 3 years ago

BTW, #14825 doesn't pause/stop any torrents. It just display torrents that can't download (due to internal upload_mode caused by disk write error) as "errored". Also it allows you to manually leave this state by pressing "Resume" button. You could see this for yourself by looking at the code changes, if you have enough knowledge of C++.

glassez commented 3 years ago

@an0n666, please participate as having experience in related issue. I'm in doubt about merging my fix into v4.3.x branch due to possible regressions.

ghost commented 3 years ago

@an0n666, please participate as having experience in related issue. I'm in doubt about merging my fix into v4.3.x branch due to possible regressions.

Sorry but it looks like OP is having a slightly different issue. He has available disk space but still getting I/O errors on some torrents.

First time I didn't check if torrent resumes automatically so I tested again. Torrent gets errored and then resumes automatically after 10 minutes which is exactly as documented in libtorrent library. However the issue with torrent not completing without force recheck still remains, and is an old issue, not a regression.

thalieht commented 3 years ago

I just tested the patch too and it works as expected.

lifeboatpres commented 3 years ago

I couldn't download a torrent at all due to this bug with 4.3.5. I installed an older 4.3.2 version to solve this problem.

JeFawk commented 3 years ago

Same issue with me with 4.3.5. There was definitely enough space. I resumed the torrent a few times and it finally finished downloading.

Blendertom commented 3 years ago

Started experiencing this as well, with v4.3.5 on Windows 10. But in my case it's to local disk, not a NAS Force resuming the torrent, temporarily starts downloading the torrent again

kevinlim512 commented 3 years ago

encountering this issue as well, even though i have 300GB free on my local disk

suprimex commented 3 years ago

Same error here, version 4.3.5, Windows 10 2004, NTFS disk 1TB, have 150Gb free.

Deiiv commented 3 years ago

Same issue, 4.3.5 on W10, 1.6TB free space, happens randomly to several torrents when running multiple at a time (not sure if relevant). Doing force resume helps in some cases (usually have to try several times)

Update: my HDD was in FAT32 format, I switched it to NTFS and it seems the issue is gone (for now). Not sure if qBitorrent has issues with FAT32 specifically

eKeiran commented 3 years ago

I've got this issue too, on Win 10, 2TB of free space, and yet. The installation from #14825 didn't work either.

vic-bay commented 3 years ago

Just got this error. I reinstalled windows recently, and tried to download stuff into existing folder. Looks like it had ownership or permissions of non-existing user from the previous windows installation. After fixing file permissions download was started normally.

FranciscoPombal commented 3 years ago

To everyone affected by this bug, please try this test build: https://github.com/qbittorrent/qBittorrent/pull/14993/checks?check_run_id=2650675200 (click the Artifacts dropdown to download it). It should print the true cause of the error.

FranciscoPombal commented 3 years ago

ping @Mike-EE

Mike-EE commented 3 years ago

@FranciscoPombal -

Forgive me, but print to where? The log file?

FranciscoPombal commented 3 years ago

@FranciscoPombal -

Forgive me, but print to where? The log file?

I believe so.

wooshkey commented 3 years ago

So I'm flying to Maui on Wed and really need to get some downloads done for a very long flight and low and behold I have the same problem. But I figured out the cause!!!

It's freaking One Drive!!!!! Somehow qBittorrent is seeing that Onedrive is full and thinks the OS is out of space. Now I hate One Drive even more! I deleted some One Drive space and the problem is solved.

Cheers gents.

Blendertom commented 3 years ago

To everyone affected by this bug, please try this test build: https://github.com/qbittorrent/qBittorrent/pull/14993/checks?check_run_id=2650675200 (click the Artifacts dropdown to download it). It should print the true cause of the error.

I viewed the log from View>Log>Show

The prints the following error: File error alert. Torrent: "TORRENT NAME". File: "FILE PATH file_open (*FILE PATH) error: The process cannot access the file because it is being used by another process

Following wooshkey's comment, I quit Onedrive to see if that was the cause, but the error still persists, even though the download folder is not on one drive.

FranciscoPombal commented 3 years ago

@Blendertom

To everyone affected by this bug, please try this test build: https://github.com/qbittorrent/qBittorrent/pull/14993/checks?check_run_id=2650675200 (click the Artifacts dropdown to download it). It should print the true cause of the error.

I viewed the log from View>Log>Show

The prints the following error: File error alert. Torrent: "TORRENT NAME". File: "FILE PATH file_open (*FILE PATH) error: The process cannot access the file because it is being used by another process

Following wooshkey's comment, I quit Onedrive to see if that was the cause, but the error still persists, even though the download folder is not on one drive.

Great, this still narrows down the issue significantly. It means some other program is opening the files that qBittorrent is attempting to write to with exclusive access. Usual suspects are anti-virus software (though at least the default Windows Defender should not be a problem), any "scanners" of some sort, and cloud synchronization clients. Try to reproduce after disabling/turning off such software.

wooshkey commented 3 years ago

If you have one drive and it is full, don't just close it, free up some space, and test again. I'm wondering if Microsoft has it so tightly integrated with the OS that just closing it is not enough. In my case it immediately stopped the error. I was getting the error on 3 different downloads.

Mike-EE commented 3 years ago

Sorry folks, but my initial complaint was/is qBt version dependent and was/is used on an MS Win10 Home system without OneDrive. My issue has always occurred when downloading (not moving) torrents to my NAS but never when downloading to local drives.

RyanTruran commented 3 years ago

I had the same issue on Windows 10, I had this issue with a 5 GB torrent downloading to local storage, a hard drive with 789 GB of free space configured as FAT32. I moved it to a different local storage drive, a hard drive with 1.6 TB of free space configured as NTFS, resumed the download and I have not seen the issue since.

I have 3 Drives and all have more than 200GB of free space.

Mike-EE commented 3 years ago

@FranciscoPombal

I think that I asked this before, but perhaps a different approach would help:

What is the signal to qBt to declare this error and pause downloading? Is qBt polling the target drive's OS for free space and, if so, what method is it using?

Blendertom commented 3 years ago

Sorry folks, but my initial complaint was/is qBt version dependent and was/is used on an MS Win10 Home system without OneDrive. My issue has always occurred when downloading (not moving) torrents to my NAS but never when downloading to local drives.

Agreed, sorry for hijacking your bug report. By any change are you getting the same error as me:

To everyone affected by this bug, please try this test build: https://github.com/qbittorrent/qBittorrent/pull/14993/checks?check_run_id=2650675200 (click the Artifacts dropdown to download it). It should print the true cause of the error.

I viewed the log from View>Log>Show

The prints the following error: File error alert. Torrent: "TORRENT NAME". File: "FILE PATH file_open (*FILE PATH) error: The process cannot access the file because it is being used by another process

Following wooshkey's comment, I quit Onedrive to see if that was the cause, but the error still persists, even though the download folder is not on one drive.

Blendertom commented 3 years ago

@Blendertom

To everyone affected by this bug, please try this test build: https://github.com/qbittorrent/qBittorrent/pull/14993/checks?check_run_id=2650675200 (click the Artifacts dropdown to download it). It should print the true cause of the error.

I viewed the log from View>Log>Show The prints the following error: File error alert. Torrent: "TORRENT NAME". File: "FILE PATH file_open (*FILE PATH) error: The process cannot access the file because it is being used by another process Following wooshkey's comment, I quit Onedrive to see if that was the cause, but the error still persists, even though the download folder is not on one drive.

Great, this still narrows down the issue significantly. It means some other program is opening the files that qBittorrent is attempting to write to with exclusive access. Usual suspects are anti-virus software (though at least the default Windows Defender should not be a problem), any "scanners" of some sort, and cloud synchronization clients. Try to reproduce after disabling/turning off such software.

The only other scanner app that I could think of was Google Drive, which I quit, and since then I've downloaded off of two different torrents, and the same one, but experienced no issue.
I also ran google drive again while downloading and still no issue.

So I doubt Google Drive was causing the error, especially given that the download folder does not upload to Drive.

I used the stable build, not the alpha that you'd shared.

If I face the same issue again I'll report back.

FranciscoPombal commented 3 years ago

@Mike-EE

I think that I asked this before, but perhaps a different approach would help:

What is the signal to qBt to declare this error and pause downloading? Is qBt polling the target drive's OS for free space and, if so, what method is it using?

We just print out the error that libtorrent gives us, really. You should use at least https://github.com/qbittorrent/qBittorrent/issues/11677#issuecomment-847364062 for testing, because I believe previously we would just print any error libtorrent gave us about such failures as "not enough space", not necessarily the exact reason.

I'm not sure how exactly libtorrent is determining the errors, but even without looking, I would be it is simply look at the return values of the appropriate system calls.

FranciscoPombal commented 3 years ago

I had the same issue on Windows 10, I had this issue with a 5 GB torrent downloading to local storage, a hard drive with 789 GB of free space configured as FAT32. I moved it to a different local storage drive, a hard drive with 1.6 TB of free space configured as NTFS, resumed the download and I have not seen the issue since.

I have 3 Drives and all have more than 200GB of free space.

Well, yeah, not being able to create/save files over 4 GiB is a known limitation of FAT32. Please don't use it in anno domini 2021 unless you really can't avoid it.

Mike-EE commented 3 years ago

@Mike-EE

I think that I asked this before, but perhaps a different approach would help: What is the signal to qBt to declare this error and pause downloading? Is qBt polling the target drive's OS for free space and, if so, what method is it using?

We just print out the error that libtorrent gives us, really. You should use at least #11677 (comment) for testing, because I believe previously we would just print any error libtorrent gave us about such failures as "not enough space", not necessarily the exact reason.

I'm not sure how exactly libtorrent is determining the errors, but even without looking, I would be it is simply look at the return values of the appropriate system calls.

I'm testing it right now, have noted several failures and this is the only information in the log:

_5/25/2021 6:38 PM - File error alert. Torrent: "XXXXXXXXX". File: "\XXXXXserver\Torrents\XXXXXXXX". Reason: XXXXXXX filewrite (\XXXXXserver\Torrents\XXXXXXX error: There is not enough space on the disk

That's the extent of the information returned. I checked Windows Event Viewer and there's no corresponding entry.

esmith13 commented 3 years ago

Having the same issue with most torrents regardless of internal HDD, USB3 HDD or Network Share (Windows SMB). Drives were all NTFS or ExFAT. Tried the test build and still error: "Couldn't write to file. The process cannot access the file because it is being used by another process. Torrent is currently in "upload only" mode. This is on Win10 Pro 20H2 build 19042.985. All drives tested had over 200GB of free space above and beyond the size of the torrent. Log states: reason: file_open error. The process cannot access the file because it is in use by another process. using the 3rd party tool "Lock Hunter" (lockhunter.com) I checked what process was locking the file immediately after getting the error and it said "No processes locking this file or folder have been found". This tool has never let me down with actual file locks in the past so I tend to believe it when it says nothing has locked the file. Also, no antivirus is installed on my test box and defender is fully disabled from any scanning or protection.

TheOv3rminD commented 3 years ago

I just had the same problem and fixed it. I t was a windows permissions issue. If you ever reinstalled your old OS, you need to the ownership of the drive or at the very least the Torrents folder. If that doesn't work then give all users permission for read / write access to the torrents folder. Also cleaning up old any SIDs on the affected drive / folder is never a bad idea. you can identify old SIDs as they will appear as a long string of characters and may have a "?" appended to them. You can use this Registry entry to add "Take Ownership" to the right click context menu: Take-Ownership.zip

PowerVANO commented 3 years ago

Happed to me today on latest Windows 10 + ReFS file system. Turned out another software was using the file.

So, the issue happens when some other software manages to open a file and qBittorrent cannot write to it.

FranciscoPombal commented 3 years ago

It seems like most people facing this issue simply have one of these problems:

Permissions issue
Filesystem issues (e.g. using FAT32)
Other programs opening files with exclusive access that qBittorrent also wants to open

None of these are issues with qBittorrent. What is a bug is the wrong message being printed (e.g. all of the above problems being reported as "not enough space on disk") - but that should be fixed with builds https://github.com/qbittorrent/qBittorrent/issues/11677#issuecomment-847364062 and later.

esmith13 commented 3 years ago

@FranciscoPombal , I agree with @Hexawolf . I have used qbittorrent on many systems for a long time and never had this issue before the last few weeks. I think my test box, which is dedicated to running only qbittorrent on Win10 x64, is a perfect example of why this IS either directly or indirectly a qbittorrent issue. I have no other software that can be conflicting installed (unless it's provided as part of Win10 of course), No antivirus installed (Defender fully disabled using services.msc and registry edits), and only NTFS (for system and some storage drives) and ExFAT (extra storage drives) formatted drives. The LockHunter app was only added after the problem started to check for file locks since I find it very reliable to use in my work as the admin of my office network. I even tried disabling drive indexing in windows in case that was somehow locking the files.

Also, your test build mentioned earlier did NOT change the error reported to anything different, it still showed as a file lock error. Using another users' suggestion of reinstalling version 4.3.2 does, however, seem to resolve the issue. Is that not proof enough that the issue lies with qbittorrent?

glassez commented 3 years ago

I have used qbittorrent on many systems for a long time and never had this issue before the last few weeks.

Maybe you just didn't notice it, because earlier errors of this kind were not displayed in the list of torrents? Actually, this is the only thing that has changed recently.

Is that not proof enough that the issue lies with qbittorrent?

No. At the very least, you should observe the qBittorrent log for some (sufficient) time to make sure that there are really no errors (or they still exist).

Mike-EE commented 3 years ago

My original issue was, and still is, that downloading to a Linux server by Windows client v4.1.9 or later will result, most of the time, in the torrent erroring and printing "There is not enough space on the disk". Version 4.1.7 in the exact same environment does not display this symptom and, to the extent that I can tell by monitoring disk activity and network traffic, there's no rejection of attempted writes by Linux to the qBt Windows client of any version and I never experience this with a local disk.

I'm not sure how the "disk locking" issue became conflated with my issue.

FranciscoPombal commented 3 years ago

@Hexawolf @esmith13

Torrent file failing for me: https://hexanet.dev/download/Test.torrent

Just tried it on a bog-standard Windows 10 64-bit installation, Windows Defender enabled, qBittorrent 4.3.5 all default settings - no such issue.

Why older versions work just perfectly? To say something like this, you need to reproduce and git bisect - so you have at least an attempt at investiation as a proof backing your statement.

That is the responsibility of the reporters. A git bisect investigation would indeed be helpful, from those who are suffering from this issue. This is looking like a tricky bug that only affects some people. Knowing exactly which change seems to be responsible for the issue should at least shed some light on the issue. Note that you would also probably need to git bisect libtorrent together with qBittorrent, since the issue might lie there.

Using another users' suggestion of reinstalling version 4.3.2 does, however, seem to resolve the issue. Is that not proof enough that the issue lies with qbittorrent?

No, but it is a good starting point to attempt an investigation with git bisect for those who are able to reproduce the bug.

@Mike-EE

My original issue was, and still is, that downloading to a Linux server by Windows client v4.1.9 or later will result, most of the time, in the torrent erroring and printing "There is not enough space on the disk". Version 4.1.7 in the exact same environment does not display this symptom and, to the extent that I can tell by monitoring disk activity and network traffic, there's no rejection of attempted writes by Linux to the qBt Windows client of any version and I never experience this with a local disk.

At this point I don't think we'll progress any further without an investigation with git bisect from your side. I recommend just using the latest RC_1_1 libtorrent when bisecting qBittorrent 4.1.x, and the latest RC_1_2 with 4.3.x. If it's still not clear where the issue is after this, you may need to bisect libtorrent RC_1_1 with qBittorrent 4.1.7. Good luck.

I'm not sure how the "disk locking" issue became conflated with my issue.

It is possible that these issues share the same underlying cause (it seems like it), it is also possible this is not the case. I think it's too early to tell. Once we know more, we can split this into 2 separate threads if that's warranted.

gxcreator commented 3 years ago

For me, it was Spotify, it seems they recently introduced bug when app scans FS for local media even if local playback is turned off. WA for spoty: enable "Files on device" and disable folders individually.

ghost commented 3 years ago

@Mike-EE

If you can confirm that the error does not happen in 4.1.7 and happens in 4.1.9, then it would be very easy to find the offending commit. Because there were only a few commits in both libtorrent and qbittorrent during these versions. In libtorrent RC1_1 there were only these two commits: https://github.com/arvidn/libtorrent/commit/a61dcdda3874ed3ff309c6e73ccd9408869cefbe https://github.com/arvidn/libtorrent/commit/909211888e7b11a67146fef117324c6a6298d46e

I would suggest you to build libtorrent R_C_1_1 with commit hash 84f10d05ca and use qBittorrent 4.1.9 to rule out if it's due to libtorrent or qBittorrent. If the culprit is libtorrent then you're left with only 2 commits to test.

qbittorrent / qBittorrent