After read/write too many data, one key-value lost.

[Eoous]

I don't know why this error happened. Is it possible?

Also i don't know whether other key-values lost.

The scenario:

1. insert key1 value1
2. insert a lot of keys and values
3. key1 and value1 lost

I guess this error happened with creating new ldb file.

[dermesser]

what is "a lot"? 10? 100? 1000?

[Eoous]

exceed 100k

[Eoous]

made a copy backup for database, i found at one point, a lot of get operations caused this, im not sure whether get operation could modify files, if i set Options::in_memory as true, this error doesn't happen

[dermesser]

thank you, I will try to reproduce it when I find some time!

[sunvim]

I test it, not happen, how do you test it ?

test code

use rusty_leveldb::{Options, DB};

fn main() {
    let mut db = DB::open("test.db", Options::default()).unwrap();

    let rs = db.put("hello".as_bytes(), "world".as_bytes());
    println!("put result: {:?}", rs);

    let rs = db.get("hello".as_bytes());

    println!("get result: {}", String::from_utf8(rs.unwrap()).unwrap());

    let mut db = insert_too_many_data(db);

    let rs = db.get("hello".as_bytes());
    println!("get result: {}", String::from_utf8(rs.unwrap()).unwrap());
}

fn insert_too_many_data(mut db: DB) -> DB {
    for i in 0..3_000_000 {
        let _ = db.put(i.to_string().as_bytes(), format!("val{}", i).as_bytes());
        let get_rs = db.get((i - 1).to_string().as_bytes());
        match get_rs {
            Some(val) => {
                let rs = String::from_utf8(val).unwrap();
                let expect = format!("val{}", i - 1);
                assert_eq!(rs, expect, "expect {} get result: {}", expect, rs);
            }
            None => {
                println!("key {} get result: None", i - 1);
            }
        }
    }
    db
}

but @dermesser is it possible to modify Read opertion condition? I agree with @0xEclair , it shouldn't need mut self

[dermesser]

Fundamentally, get() has a &mut receiver because depending on how often you read some items, it may trigger a compaction. That may also be the reason for lost items (although obviously it really shouldn't happen).

So if get() can't modify the state, it may result in sub-optimal read patterns over time.