search

qbittorrent / qBittorrent

qBittorrent BitTorrent client
https://www.qbittorrent.org
Other
27.84k stars 3.93k forks source link

Repeating crashes in case of insufficient disk space (only started after surprise disk removal) #18838

Open baccccccc opened 1 year ago

baccccccc commented 1 year ago

qBittorrent & operating system versions

qBittorrent version: v4.5.2 (64-bit) Libtorrent version: 2.0.8.0 Qt version: 6.4.2 Boost version: 1.81.0 OpenSSL version: 1.1.1t zlib version: 1.2.13 OS version: Windows 11 Version 22H2 10.0.25336 x86_64

What is the problem?

Caught signal: SIGABRT

 0# boost::stacktrace::basic_stacktrace >::init at G:\QBITTORRENT\boost_1_81_0\boost\stacktrace\stacktrace.hpp:78
 1# getStacktrace at G:\QBITTORRENT\qbt-src-4.5.2-RC_2_0\src\app\stacktrace.cpp:35
 2# `anonymous namespace'::abnormalExitHandler at G:\QBITTORRENT\qbt-src-4.5.2-RC_2_0\src\app\signalhandler.cpp:103
 3# raise at minkernel\crts\ucrt\src\appcrt\misc\signal.cpp:547
 4# abort at minkernel\crts\ucrt\src\appcrt\startup\abort.cpp:71
 5# terminate at minkernel\crts\ucrt\src\appcrt\misc\terminate.cpp:58
 6# FindHandler at D:\a\_work\1\s\src\vctools\crt\vcruntime\src\eh\frame.cpp:735
 7# __InternalCxxFrameHandler at D:\a\_work\1\s\src\vctools\crt\vcruntime\src\eh\frame.cpp:399
 8# __InternalCxxFrameHandlerWrapper at D:\a\_work\1\s\src\vctools\crt\vcruntime\src\eh\frame.cpp:234
 9# __CxxFrameHandler4 at D:\a\_work\1\s\src\vctools\crt\vcruntime\src\eh\risctrnsctrl.cpp:306
10# _chkstk in ntdll
11# RtlInitializeExtendedContext2 in ntdll
12# RtlRaiseException in ntdll
13# RaiseException in KERNELBASE
14# __ExceptionPtrRethrow at D:\a\_work\1\s\src\vctools\crt\github\stl\src\excptptr.cpp:536
15# std::rethrow_exception at C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.34.31933\include\exception:312
16# std::_Associated_state::_Get_value at C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.34.31933\include\future:299
17# QThreadCreateThread::run in qbittorrent
18# QThreadPrivate::start in qbittorrent
19# thread_start at minkernel\crts\ucrt\src\appcrt\startup\thread.cpp:97
20# BaseThreadInitThunk in KERNEL32
21# RtlUserThreadStart in ntdll

Steps to reproduce

It started happening today after I had some drive issues. The drives were surprise disconnected, and qBittorrent crashed.

Since then, all the drives have been reconnected, and the operatig system is not complaining about any filesystem issues or inaccessible files.

I cannot manually validate every file and directory (I have >40,000 torrents at this point). But at the first sight, everything seems to be in place.

However, every attempt to launch qBittorrent results in a crash on startup (shortly after "loading torrents..." window appears.)

Additional context

No response

Log(s) & preferences file(s)

qbittorrent-01.log profile-roaming.zip

baccccccc commented 1 year ago

ok, I figured that if I delete or rename torrents.db then the app starts successfully. But of course, the list of torrents is empty, and that's not what I want.

I tried installing sqlite3.exe and check the database for integrity, and it seems fine.

PS C:\Program Files\qBittorrent> & 'C:\Users\<username>\Downloads\sqlite-tools-win32-x86-3410200\sqlite3.exe'
SQLite version 3.41.2 2023-03-22 11:56:21
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> .open "C:/Users/<username>/AppData/Local/qBittorrent/torrents.db"
sqlite> PRAGMA integrity_check;
ok
sqlite> PRAGMA quick_check;
ok
sqlite> .quit
PS C:\Program Files\qBittorrent>

I'm happy to share the db with someone privately, but beware that it's ~2 GB and might contain a lot of NSFW verbatim.

baccccccc commented 1 year ago

ok, that's interesting. I cleared up some disk space on the OS drive, and now qBittorrent seems to start successfully. All of the content is there, and I don't see any issues. In fact, the app starting up slightly faster than I recall it to do before.

weird thing is that I had more or less the same amount of free disk space (~1 GB) for months, and it was not an issue before.

if qBittorrent requires a certain amount of free disk space to operate, that's perhaps understood. But hopefully we can have better indication than just crashing.

glassez commented 1 year ago

SQLite is intended to create temporary files to perform some jobs. IIRC, they can be comparable in size to the database file itself.

baccccccc commented 1 year ago

SQLite is intended to create temporary files to perform some jobs. IIRC, they can be comparable in size to the database file itself.

understandable, but does it return some meaningful status if it fails to allocate those files? If yes, we could probably intercept this and handle it gracefully (e.g., with an error message or at least a record in the log), instead of crashing.

I tried to watch app startup using Process Monitor, to see if there are any read failures. (As my hypothesis was that some of the torrent content was corrupt.) I do not recall seeing any attempts to create those temporary files. I guess SQLite queries for available space first and does not even try to create temp files if space is low. So, it should know what it's doing.

xavier2k6 commented 1 year ago

I think there's actually 2 genuine CTD scenarios at play here that I've been investigating/focusing on with lbt 2.x

  1. Insufficient disk space

0XC0000005/0XC0000006/0XE06D7363

  1. Surprise disk removal

(0xC0000028 - STATUS_BAD_STACK), doesn't seem to happen when have a debugger attached & can't reproduce reliably, eventvwr shows the error code.

Screenshot 2023-04-14 235824
baccccccc commented 1 year ago

Yeah, I apologize for mixing multiple symptoms in this bug. Originally I suspected that the following was happening.

  1. All was working fine.
  2. I had a surprise disk removal event.
  3. QBittorrent crashed.
  4. I reconnected the drives and made sure that OS and filesystems are happy.
  5. Tried to restart QBittorrent.
  6. QBittorrent was crashing every time.

To be clear, I think that a crash in case of surprise disk removal (step 3 above) is justifiable. Maybe it's not the smartest behavior, but it's definitely something unexpected and hence the crash can be understood.

However, a repetitive crash at startup (step 6), in case there are no obvious environmental issues, is a much harder pill to swallow. I had a hypothesis that something went bad, either with torrent content or torrent database, and that's what was causing the crash. I was hoping I could see the reason in either log or procmon trace, but there was nothing suspicious there.

But at the end of the day, it turned out to be not the case. Instead, that's what apparently happened.

  1. All was working fine.
  2. I had a surprise disk removal event.
  3. QBittorrent crashed.
  4. I reconnected the drives and made sure that OS and filesystems are happy.
  5. Meanwhile, free space on the OS drive has decreased slightly. (Probably because the OS was doing something, due to reboots and such.)
  6. Now I no longer had enough free space for SQLite to operate. But nobody realized that.
  7. Tried to restart QBittorrent.
  8. SQLite failed to allocate temp files
  9. QBittorrent was crashing every time.

Once I cleaned up some free space, the crashes went away. There was nothing wrong with the database or torrent content all along. So, we should stop attributing this crash to surprise disk removal. It was just a coincidence but not the real cause of the crashes.

We should realize that it was insufficient disk space (on the OS drive, not on the content drive) what was causing repetitive crashes. You can probably repro this by doing this.

  1. Enable SQLite option if not already.
  2. Make sure QBittorrent is installed on the OS drive and the database is located in the default location (under %LocalAppData% in user profile).
  3. Add a bunch of torrents to bloat up your database size. (My DB is currently 1.91 Gb. I'm not sure if it's easy to repro on a smaller DB size.)
  4. Artificially consume most of the disk space on the OS drive.
  5. Launch QBittorrent
  6. Observe crash
  7. If there's no crash and the app starts successfully, reduce the available disk space even further.

For step 4, it's easy to do so by running the following in PowerShell (if you're running Windows.)

$path = Join-Path -Path $env:temp -ChildPath ( Get-Date -Format FileDateTimeUniversal )
$size = ( Get-Partition | Where-Object -FilterScript { $psItem.isBoot } | Get-Volume ).sizeRemaining - 100mb
fsutil.exe file createNew $path $size

this will create a file under your temp directory which will leave ~100 MB free on your OS drive. If your database is tiny maybe 100 MB is not enough, and you should go for something even more aggressive (like 10 MB.) Be sure to remove it after testing :)

for the reference, here's the crash event I had in my OS log once QBittorrent was crashing due to insufficient disk space for the database.

Log Name:      Application
Source:        Application Error
Date:          4/12/2023 12:25:15 PM
Event ID:      1000
Task Category: Application Crashing Events
Level:         Error
Keywords:      
User:          <redacted>
Computer:      <redacted>
Description:
Faulting application name: qbittorrent.exe, version: 4.5.2.0, time stamp: 0x63fd4a92
Faulting module name: qbittorrent.exe, version: 4.5.2.0, time stamp: 0x63fd4a92
Exception code: 0xc0000005
Fault offset: 0x0000000000d53e74
Faulting process id: 0x0x1D6C
Faulting application start time: 0x0x1D96D7478ED949E
Faulting application path: C:\Program Files\qBittorrent\qbittorrent.exe
Faulting module path: C:\Program Files\qBittorrent\qbittorrent.exe
Report Id: cdd96c06-2583-4ecf-a17f-49930dfb3675
Faulting package full name: 
Faulting package-relative application ID:
xavier2k6 commented 1 year ago

@glassez @Chocobo1 In relation to https://github.com/qbittorrent/qBittorrent/issues/18838#issuecomment-1509376433

I believe I now have been able to reliably reproduce a CTD scenario with official qBittorrent 4.5.2/lt 2.0.8 in portable mode & will submit an Issue Report.

Edit: #19026