Unable to open Sqlite storage

[senden9]

Hi!

I can create a sqlite backend file, open some files in it, read them and close. But i am not able to reopen the same sqlite file again. Then zbox crashes with the given error below. Somehow in a frame below the RepoOpener call.

To reproduce simply execute the attached rust program two times.

It seems that I am able to use the file backend, but not the sqlite backend. If I replace sqlite://./myRepoSqlite with file://./myRepo it works. :thinking:.

use std::io::prelude::*;
use std::io::{Seek, SeekFrom};
use zbox::{init_env, OpenOptions, RepoOpener};

fn main() {
    // initialise zbox environment, called first
    init_env();

    // create and open a repository
    let mut repo = RepoOpener::new()
        .create(true)
        .open("sqlite://./myRepoSqlite", "your password")
        .unwrap();

    // List files
    let things = repo.read_dir("/").unwrap();
    println!("{:?}", things);

    {
        // create and open a file for writing
        let mut file = OpenOptions::new()
            .create(true)
            .open(&mut repo, "/my_file.txt")
            .unwrap();

        file.seek(SeekFrom::End(0)).unwrap();

        // use std::io::Write trait to write data into it
        file.write_all(b"Hello, world!\n").unwrap();

        // finish writting to make a permanent content version
        file.finish().unwrap();

        // read file content using std::io::Read trait
        let mut content = String::new();
        file.seek(SeekFrom::Start(0)).unwrap();
        file.read_to_string(&mut content).unwrap();

        println!("{}", content);
    }
}

Dependencies:

[dependencies]
zbox = {version="0.8.2", features=["storage-file", "storage-sqlite"]}

Error:

RUST_BACKTRACE=1 cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.05s
     Running `target/debug/crypting`
thread 'main' panicked at 'index out of bounds: the len is 0 but the index is 0', /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/libcore/slice/mod.rs:2695:10
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:59
             at src/libstd/panicking.rs:197
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:211
   4: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:474
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:381
   6: rust_begin_unwind
             at src/libstd/panicking.rs:308
   7: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
   8: core::panicking::panic_bounds_check
             at src/libcore/panicking.rs:61
   9: <usize as core::slice::SliceIndex<[T]>>::index
             at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/libcore/slice/mod.rs:2695
  10: core::slice::<impl core::ops::index::Index<I> for [T]>::index
             at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/libcore/slice/mod.rs:2552
  11: <alloc::vec::Vec<T> as core::ops::index::Index<I>>::index
             at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/liballoc/vec.rs:1687
  12: <zbox::volume::storage::sqlite::sqlite::SqliteStorage as zbox::volume::storage::Storable>::get_super_block
             at /home/stefano/.cargo/registry/src/github.com-1ecc6299db9ec823/zbox-0.8.2/src/volume/storage/sqlite/sqlite.rs:329
  13: zbox::volume::storage::storage::Storage::get_super_block
             at /home/stefano/.cargo/registry/src/github.com-1ecc6299db9ec823/zbox-0.8.2/src/volume/storage/storage.rs:196
  14: zbox::volume::super_block::SuperBlk::load_arm
             at /home/stefano/.cargo/registry/src/github.com-1ecc6299db9ec823/zbox-0.8.2/src/volume/super_block.rs:131
  15: zbox::volume::super_block::SuperBlk::load
             at /home/stefano/.cargo/registry/src/github.com-1ecc6299db9ec823/zbox-0.8.2/src/volume/super_block.rs:159
  16: zbox::volume::volume::Volume::open
             at /home/stefano/.cargo/registry/src/github.com-1ecc6299db9ec823/zbox-0.8.2/src/volume/volume.rs:95
  17: zbox::fs::fs::Fs::open
             at /home/stefano/.cargo/registry/src/github.com-1ecc6299db9ec823/zbox-0.8.2/src/fs/fs.rs:167
  18: zbox::repo::Repo::open
             at /home/stefano/.cargo/registry/src/github.com-1ecc6299db9ec823/zbox-0.8.2/src/repo.rs:682
  19: zbox::repo::RepoOpener::open
             at /home/stefano/.cargo/registry/src/github.com-1ecc6299db9ec823/zbox-0.8.2/src/repo.rs:247
  20: crypting::main
             at src/main.rs:10
  21: std::rt::lang_start::{{closure}}
             at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/libstd/rt.rs:64
  22: std::panicking::try::do_call
             at src/libstd/rt.rs:49
             at src/libstd/panicking.rs:293
  23: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:85
  24: std::rt::lang_start_internal
             at src/libstd/panicking.rs:272
             at src/libstd/panic.rs:394
             at src/libstd/rt.rs:48
  25: std::rt::lang_start
             at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/libstd/rt.rs:64
  26: main
  27: __libc_start_main
  28: _start

Environment information:

$ rustc -V && cargo version && uname -a && lsb_release -d
rustc 1.36.0 (a53f9df32 2019-07-03)
cargo 1.36.0 (c4fcfb725 2019-05-15)
Linux STEFANO-PC 5.1.15-arch1-1-ARCH #1 SMP PREEMPT Tue Jun 25 04:49:39 UTC 2019 x86_64 GNU/Linux
Description:    Arch Linux

Zipped Sqlite file for inspection: myRepoSqlite.zip

[burmecia]

thank you @senden9. This is a bug in sqlite storage which handles SQL statements initialization order incorrectly. I will fix it soon.

[senden9]

Hi!

If I see it right the fix is is zbox 0.8.3. If i now try to run the code I can not open the repository (or a newly created one) because of an Error on opening. So it is less worse because i can catch the error but i can still not create & open an sqlite repo.

Stack-trace of the same code as above:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: NotFound', src/libcore/result.rs:999:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:59
             at src/libstd/panicking.rs:197
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:211
   4: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:474
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:381
   6: rust_begin_unwind
             at src/libstd/panicking.rs:308
   7: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
   8: core::result::unwrap_failed
             at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/libcore/macros.rs:18
   9: core::result::Result<T,E>::unwrap
             at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/libcore/result.rs:800
  10: crypting::main
             at src/main.rs:10
  11: std::rt::lang_start::{{closure}}
             at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/libstd/rt.rs:64
  12: std::panicking::try::do_call
             at src/libstd/rt.rs:49
             at src/libstd/panicking.rs:293
  13: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:85
  14: std::rt::lang_start_internal
             at src/libstd/panicking.rs:272
             at src/libstd/panic.rs:394
             at src/libstd/rt.rs:48
  15: std::rt::lang_start
             at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/libstd/rt.rs:64
  16: main
  17: __libc_start_main
  18: _start

crypting::main line 10 is

    let mut repo = RepoOpener::new()
        .create(true)
        .open("sqlite://./myRepoSqlite", "your password")
        .unwrap();

[burmecia]

That issue is happened at the FFI boundary between rust and libsqlite. When binding eid as CString to SQL parameter, its life time isn't long enough so a dangling pointer is passed to libsqlite. It used to work on my mac, but I think it is largely by luck. I have worked out a fix, @senden9 could you try test it again and let me know if it is working? Thanks.

[senden9]

Used master branch as dependency (ed1a2cbf). Seem to work now :tada: