Database startup error: Error: SQLITE_NOTADB: file is not a database

eppfel commented 4 years ago

[x] I have searched open and closed issues for duplicates Related, but outdated: #2712 #2609

Bug Description

I just started Signal and receive the error Database startup error: Error: SQLITE_NOTADB: file is not a database with the option to delete all data and start new.

What is strange is, that I have a Signal and Signal-betafolder in ~/Library/Application Support/. The files in beta however are all created on 2020-01-08 and no changes were made after that. I thought I was using beta but somehow I did not.

Also interesting that it tries to migrate the data with each restart.

I have used Signal on and off on that machine, not on a daily basis.

Screenshots

Platform Info

Signal Version:

1.35.2

Operating System:

OS X 10.14

Linked Device Version:

4.71.2

Link to Debug Log

No link, as App did not start, so here the log file.

{"name":"log","hostname":"xBook.home","pid":16348,"level":30,"msg":"app ready","time":"2020-09-12T07:47:54.231Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16348,"level":30,"msg":"starting version 1.35.2","time":"2020-09-12T07:47:54.231Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16348,"level":30,"msg":"migrateDatabase: Migration without cipher change failed","time":"2020-09-12T07:47:54.261Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16348,"level":30,"msg":"Database startup error: Error: SQLITE_NOTADB: file is not a database","time":"2020-09-12T07:47:54.263Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16348,"level":30,"msg":"sql.initialize was unsuccessful; returning early","time":"2020-09-12T07:48:24.204Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16810,"level":30,"msg":"app ready","time":"2020-09-12T08:45:16.683Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16810,"level":30,"msg":"starting version 1.35.2","time":"2020-09-12T08:45:16.684Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16810,"level":30,"msg":"migrateDatabase: Migration without cipher change failed","time":"2020-09-12T08:45:16.698Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16810,"level":30,"msg":"Database startup error: Error: SQLITE_NOTADB: file is not a database","time":"2020-09-12T08:45:16.700Z","v":0}
{"name":"log","hostname":"xBook.home","pid":16810,"level":30,"msg":"sql.initialize was unsuccessful; returning early","time":"2020-09-12T08:45:21.291Z","v":0}

the config.js holds a key.

scottnonnenberg-signal commented 4 years ago

@eppfel When you see this error, it means that something has gone wrong with your database on disk. It happens most frequently when your computer loses power unexpectedly, or you have to shutdown your computer abruptly. Did anything like that happen recently?

eppfel commented 4 years ago

Thanks for getting back to me. I don't recall it, but the laptop is old so the battery discharges quickly.

scottnonnenberg-signal commented 4 years ago

Prior discussion about this error is here: https://github.com/signalapp/Signal-Desktop/issues/2609

eppfel commented 4 years ago

This seems different in the sense, that it fails migrating the databse in my case. Could this be related to an update?

scottnonnenberg-signal commented 4 years ago

It's definitely the same thing. We are prepared to migrate a database (encryption ciphers or database schema, as needed) on every database open.

bjdooi commented 4 years ago

I'm having the same exact issue (probably due to my computer randomly shutting off during an update). I contacted support@signal.org like the previous issues said to do but haven't heard back from them yet

primeos commented 3 years ago

This just happened to me too after a normal reboot (no crash or anything and a FS corruption seems unlikely):

{"name":"log","hostname":"jarvis","pid":15241,"level":30,"msg":"app ready","time":"2020-12-08T14:25:15.449Z","v":0}
{"name":"log","hostname":"jarvis","pid":15241,"level":30,"msg":"starting version 1.38.2","time":"2020-12-08T14:25:15.449Z","v":0}
{"name":"log","hostname":"jarvis","pid":15241,"level":30,"msg":"migrateDatabase: Migration without cipher change failed","time":"2020-12-08T14:25:15.478Z","v":0}
{"name":"log","hostname":"jarvis","pid":15241,"level":30,"msg":"Database startup error: Error: SQLITE_NOTADB: file is not a database","time":"2020-12-08T14:25:15.481Z","v":0}
{"name":"log","hostname":"jarvis","pid":15241,"level":30,"msg":"sql.initialize was unsuccessful; returning early","time":"2020-12-08T14:25:19.715Z","v":0}

I'm on GNU/Linux with Signal-Desktop 1.38.2. I'll try to find out more.

Updates:

This is the log of the last application shutdown:

{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:14:51.665Z","msg":"Sending a keepalive message","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:15:46.961Z","msg":"Sending a keepalive message","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:39.017Z","msg":"Removing notification","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:40.032Z","msg":"SQL channel job 24509 (updateConversations) succeeded in 12ms","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"msg":"close event {\"shouldQuit\":false}","time":"2020-12-07T17:16:42.391Z","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"msg":"requestShutdown: Requesting close of mainWindow...","time":"2020-12-07T17:16:42.393Z","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.077Z","msg":"Sending a keepalive message","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.396Z","msg":"background/shutdown","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.396Z","msg":"attachment_downloads/stop: disabling","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.396Z","msg":"MessageReceiver: stopProcessing requested","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.396Z","msg":"MessageReceiver.close()","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.396Z","msg":"WebSocketResource.close()","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.396Z","msg":"drained","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"time":"2020-12-07T17:16:42.397Z","msg":"MessageReceiver: unregister batchers","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"msg":"requestShutdown: Response received","time":"2020-12-07T17:16:42.449Z","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"msg":"before-quit event {\"readyForShutdown\":true,\"shouldQuit\":false}","time":"2020-12-07T17:16:42.450Z","v":0}
{"name":"log","hostname":"jarvis","pid":26735,"level":30,"msg":"close event {\"readyForShutdown\":true,\"shouldQuit\":true}","time":"2020-12-07T17:16:42.450Z","v":0}

There seems to be no indication that anything did go wrong. I've created a backup right after closing Signal-Desktop but the DB (~/.config/Signal/sql/db.sqlite) from the backup has the same hash and is also corrupt (I cannot open it manually using sqlcipher and since it's encrypted I cannot really investigate why...).

A comparison of the file sizes (last working backup vs. corrupt DB):

$ du -h Signal/sql/db.sqlite Signal-corrupt-sqlcipher-db/sql/db.sqlite
56M Signal/sql/db.sqlite
57M Signal-corrupt-sqlcipher-db/sql/db.sqlite
$ du -b Signal/sql/db.sqlite Signal-corrupt-sqlcipher-db/sql/db.sqlite
58351616    Signal/sql/db.sqlite
59084800    Signal-corrupt-sqlcipher-db/sql/db.sqlite

But only the first 16 bytes of db.sqlite seem to be identical in both files (I didn't look at the SQLCipher documentation; not sure if it should look that way or not).

ryantrinkle commented 3 years ago

I'm trying to upgrade from v1.37.3 to v1.39.4, and I"m getting this same error (NOTADB). If I manually start v1.37.3, everything still works and all data is still present.

ryantrinkle commented 3 years ago

I've now narrowed this down a bit: I was able to step through versions up to 1.38.2, which runs successfully. Running 1.39.2 (which seems to be the next release after 1.38.2) results in NOTADB.

Set Windows Application User Model ID (AUMID) { appUserModelId: 'org.whispersystems.signal-desktop' }                                              
NODE_ENV production
NODE_CONFIG_DIR /nix/store/8x3rqj3j9dg2g3f842xwymrhffvi60y6-signal-desktop-1.39.2/lib/Signal/resources/app.asar/config
NODE_CONFIG {}
ALLOW_CONFIG_MUTATIONS undefined
HOSTNAME undefined
NODE_APP_INSTANCE undefined
SUPPRESS_NO_CONFIG_WARNING undefined
SIGNAL_ENABLE_HTTP undefined
userData: /home/ryan/.config/Signal
config/get: Successfully read user config file
x-attr dependency did not load successfully
config/get: Successfully read ephemeral config file
making app single instance
{"name":"log","hostname":"aji","pid":21418,"level":30,"msg":"app ready","time":"2020-12-24T02:33:44.571Z","v":0}
{"name":"log","hostname":"aji","pid":21418,"level":30,"msg":"starting version 1.39.2","time":"2020-12-24T02:33:44.571Z","v":0}
{"name":"log","hostname":"aji","pid":21418,"level":30,"msg":"media access status undefined undefined","time":"2020-12-24T02:33:44.572Z","v":0}
{"name":"log","hostname":"aji","pid":21418,"level":30,"msg":"migrateDatabase: Migration without cipher change failed","time":"2020-12-24T02:33:44.586Z","v":0}
{"name":"log","hostname":"aji","pid":21418,"level":30,"msg":"Database startup error: Error: SQLITE_NOTADB: file is not a database","time":"2020-12-24T02:33:44.587Z","v":0}
{"name":"log","hostname":"aji","pid":21418,"level":30,"msg":"sql.initialize was unsuccessful; returning early","time":"2020-12-24T02:33:48.441Z","v":0}

scottnonnenberg-signal commented 3 years ago

@ryantrinkle It's really surprising that the changes from v1.37 to v1.39 would result in that kind of error, since no low-level database changes are included. How are you installing Signal Desktop?

primeos commented 3 years ago

I hate Signals way of dealing with long term conversations and history archival, it sucks - there are no proper ways of dealing with this. @moxie0

+1, please do something about this.

Finding and fixing the bug might be a challenge but in any case Signal-Desktop should (IMO) at least: 1) Support making backups and importing them without relying on 3rd party tools: https://github.com/signalapp/Signal-Desktop/issues/522

https://support.signal.org/hc/en-us/articles/360007059752-Backup-and-Restore-Messages ("Manually transferring folders is not supported and may result in errors. Signal uses a stateful protocol, meaning that you cannot simply copy files to another instance of Signal or copy files after linking and using Signal.")
3rd party: https://github.com/NixOS/nixpkgs/issues/106038 2) Support fetching(/importing) older messages from the phone (if the user wants it): https://github.com/signalapp/Signal-Desktop/issues/1651

neonfuz commented 3 years ago

I have been getting network errors in signal today so I looked at issues for my distro (nixos) and found this bug, so I quit signal to backup my signal folder, and upon re-opening it I had a corrupted database... very annoying, my phone is broken right now so its going to be a pain getting access to signal now.

fdietze commented 3 years ago

I have the same problem. So what are my options now? Is there a way repair the corrupted database? Do I have to delete my history? Should I wait for a fix?

backuitist commented 3 years ago

You could check the first 16 bytes of your db file:

$ hexdump -C ~/.config/Signal/sql/db.sqlite | head -1

You're supposed to see this:

00000000  53 51 4c 69 74 65 20 66  6f 72 6d 61 74 20 33 00  |SQLite format 3.|

On my broken DB I had this:

00000000  f0 fd 28 28 7e 5b 5a d7  8f 3f d1 ea 60 73 1c 47  |..((~[Z..?..`s.G|

I couldn't find any tool able to repair a DB with a bad header...

fdietze commented 3 years ago

Hmm, I have:

00000000  f7 93 95 e1 ef b4 ad ee  98 3b ac 32 54 15 84 10  |.........;.2T...|

:disappointed:

mehdibo commented 3 years ago

Same error just happened to me today

Database startup error:

Error: SQLITE_NOTADB: file is not a database

fdietze commented 3 years ago

Since I wanted to use Signal on my Desktop again, I just deleted my whole chat history. That felt really bad...

panicfarm commented 3 years ago

This Database startup error: Error: SQLITE_NOTADB: file is not a database on Windows 10 makes the program unusable- it has destroyed my chat history on two occasions. It only happens after the app autoupdate, when I quit and restart the same version of Signal, the DB is opened fine. When I chose "Delete All Data" on the error dialog, it shows


Error: EBUSY: resource busy or locked, unlink 'C:\Users\[REDACTED]\AppData\Roaming\Signal\sql\db.sqlite'
    at Object.unlinkSync (fs.js:930:3)
    at Function.rimrafSync [as sync] ([REDACTED]\app.asar\node_modules\rimraf\rimraf.js:306:17)
    at removeDB ([REDACTED]\app.asar\app\sql.js:1150:10)
    at Object.initialize ([REDACTED]\app.asar\app\sql.js:1123:13)

Using Process Explorer, I could not find a process that was locking db.sqlite

norpol commented 3 years ago

This issue is so painful. There are journalists on Twitter reporting that they've lost access to their Signal app & sources due to this bug.

sonergonul commented 3 years ago

Now, this is on Hacker News: https://news.ycombinator.com/item?id=26292299

justinclift commented 3 years ago

@backuitist If Signal is using SQLCipher, as mentioned earlier in this thread, then the header of the encrypted database isn't the standard SQLite header.

For example, this is the header from a SQLCipher database with some generated ~random data for testing purposes:

$ hexdump -C 2.5mb-encrypted-abc123.sqlite | head -1
00000000  ff 3f fc f2 74 1a 61 44  5c c7 44 0e bb 0b a1 42  |.?..t.aD\.D....B|

That database file opens fine with any client that can use SQLCipher version 4.x. eg DB Browser for SQLite

Note, this is the above test database itself (not a Signal one), just in case it's useful for someone to investigate with. Password is "abc123":

2.5mb-encrypted-abc123.zip <-- thrown inside a .zip file, so GitHub would accept the upload

justinclift commented 3 years ago

@ryantrinkle

I'm trying to upgrade from v1.37.3 to v1.39.4, and I"m getting this same error (NOTADB). If I manually start v1.37.3, everything still works and all data is still present.

That kind of sounds like the newer version(s) of Signal is using different encryption settings than the older version. Would personally kind of expect the newer version to at least try reading in the "old" database using the "old" settings, and then write out the new database using the "new" settings. Maybe something is going wrong with that process?

@norpol Sounds like the journalists losing their data should revert to the older Signal version until this problem is resolved. Hopefully that's something they can do.

norpol commented 3 years ago

@justinclift Thanks, I'll forward this information. Update: Suddenly just asking to re-link, so the archive is thankfully not gone.

Regarding SQLCipher At least for the my current version (1.39.6) the header of the database is 00000000 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 |SQLite format 3. and using sqlite3-cli it allows me to see actual message contents - just by opening the database. If SQLCipher was in use I wouldn't see that header/be able to extract the database I assume.

justinclift commented 3 years ago

@norpol Yeah, that sounds like the database you're trying there isn't encrypted. At least, not with SQLCipher.

backuitist commented 3 years ago

Thanks @justinclift good to know! I thought there would be some sort of encryption but couldn't find anything in the codebase, and the signal error message was exactly matching that of SQLite.

gkft commented 3 years ago

My signal-desktop (1.40.1) is affected by the same bug, but what looks pretty strange to me the encrypted database opens perfectly fine with sqlcipher and no corruption at all.

sqlcipher --version
3.33.0 2020-08-14 13:23:32 fca8dc8b578f215a969cd899336378966156154710873e68b3d9ac5881b0alt1 (SQLCipher 4.4.2 community)
sqlcipher db.sqlite "PRAGMA key = \"x'$(jq -r '.key' config.json)'\"; attach database 'plaintext.db' as plaintext key ''; SELECT sqlcipher_export('plaintext'); DETACH DATABASE plaintext"

as proposed in https://github.com/signalapp/Signal-Desktop/issues/522#issuecomment-787527821

Maybe the affected version of signal assumes another encryption scheme of the database, a indicator maybe migrateDatabase, which is output from here https://github.com/signalapp/Signal-Desktop/blob/a173a9b33ff8dccbe02b9ad2d5e268ca3b036563/ts/sql/Server.ts#L329

And here the log:

{"name":"log","hostname":"host","pid":6857,"level":30,"msg":"app ready","time":"2021-03-13T16:54:37.550Z","v":0}
{"name":"log","hostname":"host","pid":6857,"level":30,"msg":"starting version 1.40.1","time":"2021-03-13T16:54:37.550Z","v":0}
{"name":"log","hostname":"host","pid":6857,"level":30,"msg":"media access status undefined undefined","time":"2021-03-13T16:54:37.551Z","v":0}
{"name":"log","hostname":"host","pid":6857,"level":30,"msg":"migrateDatabase: Migration without cipher change failed","time":"2021-03-13T16:54:37.563Z","v":0}
{"name":"log","hostname":"host","pid":6857,"level":30,"msg":"Database startup error: Error: SQLITE_NOTADB: file is not a database","time":"2021-03-13T16:54:37.563Z","v":0}
{"name":"log","hostname":"host","pid":6857,"level":30,"msg":"sql.initialize was unsuccessful; returning early","time":"2021-03-13T16:54:59.511Z","v":0}

Also I tested if re encrypting the database using sqlcipher with no effect.

@scottnonnenberg-signal any updates on this?

WildPenquin commented 3 years ago

I was hit by this problem today, too.

The computer is mostly stable, there has been one (1) hard crash during the last month, but IIRC, Signal was not running at the time (I'm not sure when the "corruption" of the db happened). Either way, a user should not be faced with a situation where only re-linking is possible, and losing all conversation history on the computer. This is also an accessibility problem, since some users might be more dependent on the desktop software (since desktop computers are usually more accessible than smartphones).

This is the terminal output I got when I was hit by this when starting signal-desktop (1.40.1) on Arch Linux:

$ signal-desktop 
Set Windows Application User Model ID (AUMID) { appUserModelId: 'org.whispersystems.signal-desktop' }
NODE_ENV production
NODE_CONFIG_DIR /usr/lib/signal-desktop/resources/app.asar/config
NODE_CONFIG {}
ALLOW_CONFIG_MUTATIONS undefined
HOSTNAME undefined
NODE_APP_INSTANCE undefined
SUPPRESS_NO_CONFIG_WARNING undefined
SIGNAL_ENABLE_HTTP undefined
userData: /home/ville/.config/Signal
config/get: Successfully read user config file
x-attr dependency did not load successfully
config/get: Successfully read ephemeral config file
making app single instance
{"name":"log","hostname":"ArkkiVille","pid":10839,"level":30,"msg":"app ready","time":"2021-03-13T19:57:16.402Z","v":0}
{"name":"log","hostname":"ArkkiVille","pid":10839,"level":30,"msg":"starting version 1.40.1","time":"2021-03-13T19:57:16.402Z","v":0}
{"name":"log","hostname":"ArkkiVille","pid":10839,"level":30,"msg":"media access status undefined undefined","time":"2021-03-13T19:57:16.402Z","v":0}
{"name":"log","hostname":"ArkkiVille","pid":10839,"level":30,"msg":"migrateDatabase: Migration without cipher change failed","time":"2021-03-13T19:57:16.417Z","v":0}
{"name":"log","hostname":"ArkkiVille","pid":10839,"level":30,"msg":"Database startup error: Error: SQLITE_NOTADB: file is not a database","time":"2021-03-13T19:57:16.418Z","v":0}
{"name":"log","hostname":"ArkkiVille","pid":10839,"level":30,"msg":"sql.initialize was unsuccessful; returning early","time":"2021-03-13T19:57:24.718Z","v":0}

Moreover, "copy error and quit" does not actually seem to copy the error - I'm presuming it should copy it to clipboard. If it doesn't - and if that's not what it is supposed to do, the dialog should be more clear since I have no idea where it is copying the error (message?).

I've copied the old signal configuration directory into a safe place in case it might prove useful.

joukewitteveen commented 3 years ago

I encountered the same thing on Arch Linux. When I downgraded gtk3 to version 3.24.27-3, the error went away. The only change is that tracker3 got enabled.

johackim commented 3 years ago

I have the same problem on Arch Linux since I upgrade my system. I have a backup of the ~/.config/Signal directory but even if I restore it, it doesn't work. Any idea ?

EDIT: It work when I downgrade gtk3 :)

kevenwyld commented 3 years ago

@joukewitteveen

I encountered the same thing on Arch Linux. When I downgraded gtk3 to version 3.24.27-3, the error went away. The only change is that tracker3 got enabled.

Can confirm, downgrading gtk3 on archlinux resolves the issue. For me I went to gtk3-1:3.24.26-1 however since it was the last release (not just a PKGBUILD change). I have no idea why this works yet though.

richardschuetz commented 3 years ago

@joukewitteveen

I encountered the same thing on Arch Linux. When I downgraded gtk3 to version 3.24.27-3, the error went away. The only change is that tracker3 got enabled.

Can confirm, downgrading gtk3 on archlinux resolves the issue. For me I went to gtk3-1:3.24.26-1 however since it was the last release (not just a PKGBUILD change). I have no idea why this works yet though.

The tracker3 search engine loads SQLite into to process. Signal already loads SQLCipher (specialized build of SQLite with encryption). Both apparently do not play well together. The possibility of this leading to problems was also mentioned by a SQLCipher developer some time ago: https://discuss.zetetic.net/t/is-the-multiple-sqlite-problem-an-issue-for-sqlcipher-for-android/1449/2

There is already a similar problem with OpenSSL discussed in https://github.com/signalapp/Signal-Desktop/issues/2634.

justinclift commented 3 years ago

The tracker3 search engine loads SQLite into to process. Signal already loads SQLCipher ...

Ouch. That's not good.

Sounds like the "tracker3" thing might need changing so it loads the SQLCipher library instead. Then there wouldn't be a conflict.

Maybe screwing with LD_LIBRARY_PATH could make that happen, as a workaround anyway?

devansh08 commented 3 years ago

@joukewitteveen

I encountered the same thing on Arch Linux. When I downgraded gtk3 to version 3.24.27-3, the error went away. The only change is that tracker3 got enabled.

Can confirm, downgrading gtk3 on archlinux resolves the issue. For me I went to gtk3-1:3.24.26-1 however since it was the last release (not just a PKGBUILD change). I have no idea why this works yet though.

On Arch Linux as well, downgrading to gtk3-1:3.24.27-2 worked for me. I was already on gtk3-1:3.24.27-3 when Signal threw this error for me.

gkft commented 3 years ago

I can also confirm, downgrading gtk3 on archlinux resolves the issue. gtk3 (1:3.24.27-3 -> 1:3.24.26-1)

Mushoz commented 3 years ago

Ran into the same issue with Arch today. Can confirm that downgrading gtk3 fixes the issue.

fkroener commented 3 years ago

Ran into the same issue. Decided I didn't want to delete my data and investigate first whether there was a solution.

Clicked the "X" in the window decoration of the error popup - all my data got purged. Thanks a lot Signal team for your excellent error handling and backup strategy.

MatejLach commented 3 years ago

Can confirm downgrading works but is at best a temporary solution. Does anyone have any links as to why this was enabled now, are there any bug reports on the Arch bug tracker related to this yet?

real-or-random commented 3 years ago

Can confirm downgrading works but is at best a temporary solution. Does anyone have any links as to why this was enabled now, are there any bug reports on the Arch bug tracker related to this yet?

https://bugs.archlinux.org/index.php?do=details&task_id=69980

No need to ask here, this was easy to find...

gkft commented 3 years ago

Maybe the observed behavior is also the reason for https://github.com/signalapp/Signal-Desktop/issues/5097 as now out of a sudden the database is created unencrypted i.e. sqlite instead of the bundled sqlcipher ist used.

aurelg commented 3 years ago

Maybe the observed behavior is also the reason for #5097 as now out of a sudden the database is created unencrypted i.e. sqlite instead of the bundled sqlcipher ist used.

If I understand correctly, a package unrelated to signal-desktop could be able to break it in a way that basically leads to 1/ the loss of existing history, and 2/ history being unencrypted from now on.

If that's the case, I think it is a major security issue: a lack of integrity breaks availability and confidentiality (ref here).

In addition, the benefit of using sqlcipher (with the encryption key in cleartext next to the database) vs sqlite3 should be discussed. Is that documented somewhere?

gkft commented 3 years ago

Well it is just an educated guess from my side at least it could be a possible explanation for https://github.com/signalapp/Signal-Desktop/issues/5097 https://github.com/signalapp/Signal-Desktop/issues/4513. If the database is encrypted and the key is stored next to the database this doesn't provide additional security, maybe this is for future use like having a local password for signal desktop as the key of the database can be exchanged.

LukeLR commented 3 years ago

I also lost thousands of messages due to this bug. I can confirm that after upgrading to gtk 3.24.27, Signal creates a new plaintext database. After downgrading to 3.24.26 again, Signal creates an encrypted database. I am not sure what to do, as I don't want to stick with the older GTK version, but I also don't want to loose all my messages again when the bug is fixed and Signal expects an encrypted database again. Any ideas?

gkft commented 3 years ago

The dependency of sqlite is caused by tracker3 which requires sqlite. Maybe installing tracker3 only to the local pacman database and not actually to the filesystem fixes this (does not work). Based on a reddit post

sudo pacman -S tracker3 --dbonly

And adding tracker3 to IgnorePkg list in /etc/pacman.conf as well to avoid updates. This also may break things.

Until this is fixed I will stick to sudo pacman -Syu --ignore gtk3

MatejLach commented 3 years ago

The dependency of sqlite is caused by tracker3 which requires sqlite. Maybe installing tracker3 only to the local pacman database and not actually to the filesystem fixes this (untested). Based on a reddit post

sudo pacman -S tracker3 --dbonly

And adding tracker3 to IgnorePkg list in /etc/pacman.conf as well to avoid updates. This also may break things.

Until this is fixed I will stick to sudo pacman -Syu --ignore gtk3

Already tested. Doesn't work because tracker3 libs are now required at runtime.

kpcyrd commented 3 years ago

Arch maintainer here, we've confirmed the issue and filed a bug on the gtk3 package: https://bugs.archlinux.org/task/69990

Jaakkonen commented 3 years ago

FYI: Pressing Alt+F4 on the error dialog causes DB deletion. Not happy from finding that out...

seadra commented 3 years ago

@Jaakkonen Happened to me as well (I clicked on the close window button) , IMHO Signal Desktop needs a better confirmation UI before performing the drastic action of deleting the database.

buzo-ffm commented 3 years ago

@Jaakkonen Happened to me as well (I clicked on the close window button) , IMHO Signal Desktop needs a better confirmation UI before performing the drastic action of deleting the database.

Even worse, the popup nowhere states that it's coming from Signal Desktop. Today I logged in after an Arch Linux update, several programs started automatically as usual, and then I saw this popup and wondered where it is coming from. Only after I searched the internet for this error message I knew that it's Signal Desktop and that I should not click on the “delete” button. Please fix this UI!

BTW, downgrading gtk3 to 1:3.24.27-2 fixed it for me, too. There is no need to go back to 1:3.24.26-1.

justinclift commented 3 years ago

As a reminder to everyone hitting this... please back up your database, in whatever shape it's in (encrypted or not).

It looks like many of the "lost all my messages" problems can probably be resolved. But that'll only work if you still have your old database file to work with. :smile:

kpcyrd commented 3 years ago

@justinclift the signal protocol is very stateful due to the ratchet design, rolling back to an old database is likely going to cause issues with encrypted chat sessions.

signalapp / Signal-Desktop