BiglySoftware / BiglyBT

Feature-filled Bittorrent client based on the Azureus open source project
https://www.biglybt.com
GNU General Public License v2.0
1.59k stars 153 forks source link

BiglyBT is allocating when I had turned that off #2375

Closed as-muncher closed 10 months ago

as-muncher commented 2 years ago

Java 1.8.0_202 (64 bit) Oracle Corporation c:\program files\biglybt\jre

SWT v4942r22, win32, zoom=100, dpi=120 Windows 10 v10.0, amd64 (64 bit) B2.9.0.1_B15/4 az3

When I'm shutting the software down, BiglyBT puts all the torrents into a stopped state, but when starting them up, allocates, then checks, although it's a quick check usually, then queues them. I have allocation turned off. I have sparse files mode turned on, and I don't like a bunch of incomplete files on my hard drive when they actually have no data. This happens when my computer is in airplane mode, so it's not that data is being downloaded and that's why I'm seeing all these allocated files with the .!qB extension.
biglybt allocating when not supposed to biglybt options 2 biglybt more options biglybt allocate is off

ferdnyc commented 2 years ago

I have allocation turned off.

You have complete pre-allocations turned off — if you turned them on, then BiglyBT would create a full-size, zeroed file for each download target. Because that's disabled, it only allocates the space it needs.

Pre-allocations aren't all allocations. Allocations can't be completely turned off, they're part of the file-creation process.

I have sparse files mode turned on, and I don't like a bunch of incomplete files on my hard drive when they actually have no data.

But sparse mode has nothing to do with that. All sparse mode means is that you don't have to pad a file out to full-size, when it contains mostly empty space. That comes in handy when only the last piece of a file needs to be downloaded. Without sparseness, a 50MB file with only 256K of data right at the end would have a size of 50MB, and the first 40.74MB 49.74MB would be filled with 0s. Sparseness allows that same file to have a size of 256K. But it has nothing to do with the existence of the file itself.

When a torrent is started, enabled files are opened for writing. ("Enabled files" includes border files which may themselves be set "do not download", but have their first/last piece enabled because they share that piece with an enabled file.) That creates, if nothing else, a 0-byte file on disk for each enabled output. That's just how it works. The data's got to go somewhere after it's received. Download speeds would plummet, if each time a piece is received there had to be a check for an existing file, a creation if it didn't exist, and a space allocation for the piece. That's a lot of "stuff" to do in the middle of receiving data. (Not to mention, there's way more potential for collisions when two or more pieces of the same file are received at virtually the same moment.)

That "Limit active major disk operations..." checkbox is probably the only reason you can even see the allocation phase on startup. Normally it goes by too quick to even notice, since all it has to do is open a bunch of files and (possibly) seek to the start of the active data region.

If those same torrents were just active, a lot of that can probably be done from the disk cache. But if the allocations have to wait in line with piece re-checks, that'll cause delays. (And possibly cache misses, as the data for previously-opened files is edged out by re-checked pieces being read in.)

Also depends what type of media you're using. With an SSD, there's no reason at all to make disk operations queue up and take turns. With spinning rust, there's a little more room to debate the benefits tradeoff.

as-muncher commented 2 years ago

@ferdnyc Thanks for the response. I think, though, at one time, that BiglyBT used to just write files as the data came in, and didn't pre-allocate the files, meaning, that I did not see all these files on my hard drive with X bytes, but 0 bytes on disk. Plus, BiglyBT, when doing piece mode copy, used to also copy over incomplete files to be checked, which was kind of good in a sense, just to get more data, but because a file was only 10 kb on disk, but when finished is 1 GB, that 1 GB would get copied.

as-muncher commented 2 years ago

Java 1.8.0_202 (64 bit) Oracle Corporation c:\program files\biglybt\jre

SWT v4942r22, win32, zoom=100, dpi=120 Windows 10 v10.0, amd64 (64 bit) B3.0.0.1_B03/4 az3

biglybt still allocating, meaning still making all these shell files. Maybe it has to do with the fact that I make it add the .!qB extension? I have it so that it can create files piecemeal, whatever that's called. I would rather only see files created when there's actually data being written, not empty shell files.

as-muncher commented 2 years ago

My mistake. It's not creating all the shell files, just a few. But with a big torrent, BiglyBT does this "allocating" thing with it for a long time.

as-muncher commented 2 years ago

BiglyBT is still allocating, but perhaps the area to look at is that I'm saving with the .!qB extension?? Maybe the code still allocates because I use an extension for not-done files.

as-muncher commented 2 years ago

Java 1.8.0_202 (64 bit) Oracle Corporation c:\program files\biglybt\jre

SWT v4942r22, win32, zoom=100, dpi=120 Windows 10 v10.0, amd64 (64 bit) B3.0.0.1_B29/4 az3

I'm finding that BiglyBT is not only allocating all the files with a .!qB extension, which I asked for, but that also the .jpg files are being allocated additionally with no .!qB extension. I'm not sure if this is because when my .torrent save window pops up, I deselect all, then in the search box, enter "jpg", and then mark all the .jpg files for download. So, to recap, all the files in my torrents are being saved as empty files with the .!qB extension, including the .jpg files, and the .jpg files also are being saved as empty files with no .!qB extension, so double allocating for the .jpg files, one with the .!qB extension and one without. Also, when I resume my torrents from a paused state, then biglybt spends time "allocating".

realkot commented 11 months ago

Any solution? I have slow SD card and BiglyBT allocate all torrents files that starts to download...

parg commented 11 months ago

Use "sparse files" - Options->Files: Use sparse files when supported by the file system

as-muncher commented 11 months ago

ok, with sparse files turned on, biglybt isn't allocating anymore. Ok, thanks.

as-muncher commented 10 months ago

I'm wrong - BiglyBT does allocate, but it happens after a torrent starts up after being queued, or when biglybt restarts.

parg commented 10 months ago

don't reopen things I have closed

as-muncher commented 10 months ago

@parg, Actually, if I create an Issue, I have the ability to close it and reopen it, as do you. But if you close an issue before I do, then I can't reopen it.