Excessive use of space on disk

[pdamoc]

qBittorrent & operating system versions

qBittorrent: 4.4.2 Operating system: macos 12.2.1

What is the problem?

There is a 10x difference between the size of the file downloaded and the on-disk representation of the said file.

For example, a 3GB .mkv file will use 30GB of on disk storage.

Steps to reproduce

1. Download torrent that contains a large file. For example: https://releases.ubuntu.com/21.10/ubuntu-21.10-desktop-amd64.iso.torrent
2. In Finder, right click on resulting file and select Get Info (⌘+I).
3. Look at he difference between size and size on disk.

Additional context

Here is a screenshot for 4.3.9 (only twice the size)

Here is a screenshot for 4.4.2 (10x the size)

Please note that the Prealocate disk space has been checked and in the second context I did not wait for the torrent to finish.

Log(s) & preferences file(s)

No response

[ghost]

@arvidn ping!

[arvidn]

Do you know what that extra space is used for? Normally, size-on-disk is less than the file size, for sparse files. It's unintuitive that it would be greater. The only reason I can think of why size-on-disk would be greater is if TimeMachine is taking many snapshots of the file. Every page that isn't fully sparse but changes during the download could potentially be saved in a TimeMachine snapshot.

Another possibility would be that the file is more fragmented, but it seems unlikely the filesystem would be that bad.

[pdamoc]

The two versions I tried (4.3.9 and 4.4.2) behave differently and if I try to download the same torrent with Transmission, I get a size on disk similar to the actual size of the file.

This happens will all other things staying the same.

Time Machine is disabled on my machine and even when it was enabled back in 2017, the download folder was excluded from backups.

[pdamoc]

I deleted all my configuration files and re-downloaded the same torrent. It looks like the size problem appears only if I check Pre-allocate disk space for all files.

Here is a normal behavior without preallocation.

[ghost]

Maybe related issues https://github.com/qbittorrent/qBittorrent/issues/15942

[Chocobo1]

Here is a screenshot for 4.4.2 (10x the size) Screenshot 2022-03-31 at 16 44 56

It is strange that it calculated to 31.21 GB while the bytes are correct and consistent between screenshots (3,116,482,560). Could it be a quirk in macOS?

[ghost]

@Chocobo1 one is file size another is size on disk.

[allenfengjr]

same problem, and the size of the file size increases with the download. I don't understand why it changes after I pre-allocate disk space, it is weird.

[arvidn]

@allenfengjr is this also on APFS? APFS has a feature of snapshots of files and the whole filesystem (I believe). If there's a snapshot of the whole system, every disk block becomes copy-on-write. Writing to such a file will need to allocate a new block for the new data.

I suspect this is the underlying cause of the file using so much space, that there are older snapshots of it locking the previous data on disk.

[ghost]

@arvidn the issue doesn't exist with libtorrent 1.2.15. It's a RC2_0 issue. Does snapshots have anything to do with it?

[arvidn]

I can't explain why the size on disk would balloon like that. This effect is beyond the API/contract with the operating system. It's likely caused by some subtle implementation detail in either the operating system or the filesystem. It would need some investigation.

I suspect there are tools that can be used to examine the file, and compare it to the expected file, to give some hints of what's going on. I wouldn't really expect space used by older snapshots of the file to count against its size-on-disk either. perhaps the number of extents in the file is really large, and so the metadata is using up all that extra space. But even that seems unreasonable.

[ghost]

Check this issue as well. It seems to be related with mmaped disk I/O. https://github.com/qbittorrent/qBittorrent/issues/16843

[LitoFrame]

I have the same problem. I had 171 GB free space of my HDD when I tried to download 31 GB torrent. I got I/O Error message about insufficient space on the drive. Finally 184 GB space needed for 31 GB.

[itseramin]

Same problem

[IAmOrion]

Here in November 2022 with the same issues!! A 1.5GB file is 40GB "on disk"! Tried downloading something at 30GB and the "size on disk" was almost 280GB and crashed almost every open app as the disk was "full" and showing about 10mb free. That was not a pleasant time recovering that as the system was going crazy. Another download is 26GB but taking up 138GB on disk! This is insane!!

How are people reclaiming the disk space!? I've closed qbittorrent, I've made sure pre-allocate disk space is OFF, I've ran cleaner tools such as CleanMyMacX, I've rebooted (multiple times!) and yet there's still a stupid amount of disk space being held at ransom apparently by files downloaded via this program!?

Update: I turned OFF pre-allocation etc, rebooted, managed to reclaim disk space by simply copy & pasting the file inside the same folder on the HDD and then deleting the downloaded one. Copying seems to correct the size.

Just started a new torrent as a test... within 5 mins 10GB torrent is using 138GB of disk space!

[EdKFL]

I have the same issue on Windows 11. I've seen the problem even if it hasn't become an issue for me (16TB NAS), but it will eventually. It has to be with the software if it's crossing OSs

[arvidn]

it would be helpful if someone experiencing this would examine the file (e.g. the number of extents) to try to understand why it's using so much space, and reporting back here.

Does it not happen if pre-allocate is disabled? Are there some other setting that triggers the issue?

[naks110]

I am storing incomplete download on another hdd partition- NTFS. Had 32 gb left, downloading something of 25 gb & it fills up the temp partition - 0 free space shown in explorer.

Then I run chkdsk /f /x d: - and again some free space like 13gb is shown. Then qbit can resume without errors.

pre-allocate is off, ntfs files compression enabled.

Also note, even tho after completion it's supposed to move the files to another partition/location - I have seen occassionaly it does not delete the files/folder from the temp storage.

[IveanEx]

@naks110 I encountered exactly the same problem. After running chkdsk, the free space is back. However, the check tools in the properties of the drive cannot find this problem. It looks like a bug related to ntfs file compression, as I also encountered that in other software.