no such table: main.sms_fts

[Pi991]

I've run following command (original password removed):

$ signal-backup-decode signal-2018-06-18-00-30-12.backup --output-path ./output --sqlite-path ./output/signal_backup.db --password <some_password>

Unfortunately I'm getting error as below:

Database Version: 9ed 1 frames, 0 attachments, 54 bytes into file thread 'main' panicked at 'Failed to prepare statement: INSERT INTO sms VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?): Error { code: Some(1), message: Some("no such table: main.sms_fts") }', libcore/result.rs:945:5 stack backtrace: 0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49 1: std::sys_common::backtrace::print at libstd/sys_common/backtrace.rs:71 at libstd/sys_common/backtrace.rs:59 2: std::panicking::default_hook::{{closure}} at libstd/panicking.rs:211 3: std::panicking::default_hook at libstd/panicking.rs:227 4: std::panicking::rust_panic_with_hook at libstd/panicking.rs:463 5: std::panicking::begin_panic_fmt at libstd/panicking.rs:350 6: rust_begin_unwind at libstd/panicking.rs:328 7: core::panicking::panic_fmt at libcore/panicking.rs:71 8: core::result::unwrap_failed 9: signal_backup_decode::decode_backup 10: signal_backup_decode::main 11: std::rt::lang_start::{{closure}} 12: std::panicking::try::do_call at libstd/rt.rs:59 at libstd/panicking.rs:310 13: __rust_maybe_catch_panic at libpanic_unwind/lib.rs:105 14: std::rt::lang_start_internal at libstd/panicking.rs:289 at libstd/panic.rs:374 at libstd/rt.rs:58 15: main 16: __libc_start_main 17: <unknown>

Rustc version: rustc 1.27.0 (3eda71b00 2018-06-19)

Any help greatly appreciated.

[jaimet]

I'm getting the same problem. I don't fully understand what the cause is, but here's what I've discovered so far...

The table referenced ("main.sms_fts") is part of the (new) "Full text search" facility. The contents of the SMS and MMS full-text-search tables are not exported/imported (see https://github.com/signalapp/Signal-Android/blob/master/src/org/thoughtcrime/securesms/backup/FullBackupExporter.java#L86) - instead, I think that these tables are populated using triggers that fire upon any changes (inserts/updates/deletes) in the SMS table (see https://github.com/signalapp/Signal-Android/blob/master/src/org/thoughtcrime/securesms/database/SearchDatabase.java#L32 for the "After Insert" trigger).

Unfortunately, my rust isn't good enough to completely follow main.rs, but I'm guessing that the trigger is being created while the destination table isn't being created, then when a row is inserted into the "sms" table, sqlite barfs...

HTH, :slightly_smiling_face:

[pajowu]

firstly, thank you for the issue. The new Full Test Search functionality creates a lot of issues with backups. It broke restores in the offical app for a bit (see https://github.com/signalapp/Signal-Android/issues/7863).

I pushed an update with commit 3e8fcfb, version 0.1.3. It would be great if you could test this version and report back if it works

[jaimet]

Ok, cargo install signal-backup-decode said it was going to download v0.1.2 and I've just checked https://crates.io/crates/signal-backup-decode and that page agrees (it, too, says that the latest version is 0.1.2), so presumably I need to "git clone" and compile from source...

I then tried: git clone https://github.com/pajowu/signal-backup-decode followed by: cd signal-backup-decode/ then: cargo build but this returned:

error[E0580]: main function has wrong type
   --> src/main.rs:330:1
    |
330 | fn main() -> Result<()> {
    | ^^^^^^^^^^^^^^^^^^^^^^^ expected (), found enum `std::result::Result`
    |
    = note: expected type `fn()`
               found type `fn() -> std::result::Result<(), errors::Error>`
error: aborting due to previous error
error: Could not compile `signal-backup-decode`.
To learn more, run the command again with --verbose.

I've checked line 330 in main.rs and it appears that it is identical to the last version which ran successfully (v0.1.1), so I'm assuming it's my build environment that's borked. Any ideas? (Remember, I know nothing about rust!) :slightly_smiling_face:

[pajowu]

You need at least rust version 1.26. The new version should be on crates.io now, I made a mistake uploading it

[jaimet]

Success!! Unfortunately debian stable ("stretch") contains a version of the rust compiler that is too old, so I had to find another machine, install debian testing ("buster") and try it on that. It's worth noting that in addition to a "standard install", I also had to install the "pkg-config" and "libssl-dev" packages, hence:

sudo apt install pkg-config libssl-dev cargo

before the: cargo install signal-backup-decode

The decode works perfectly. One tiny thing to correct: as the output status lines don't contain carriage returns, the last line of output is mangled, thus:

Database Version: 7ed 1 frames, 0 attachments, 54 bytes into file
Moved sqlite to decoded.dbrames, 2 attachments, 38585 bytes into file

(My output database name for the above command is decoded.db).

Thanks again! :slightly_smiling_face: :+1:

[pajowu]

Great, thank you.

The missing newlines are a known problem. I didn't yet have an idea for a better solution because if i add newlines, the status lines will spam the log (its one for ever 10 messages i think)

[jaimet]

Re the newlines, how about putting a newline at the beginning of the string on line 427, thus: "Moved sqlite to {}" becomes "\nMoved sqlite to {}" (I hope my interpretation of https://doc.rust-lang.org/reference/tokens.html#string-literals is correct!) I have not, however, worked out which other statements would also need to be modified (it might just be 431 and 438). :slightly_smiling_face: