streamingfast / substreams-rs

7 stars 3 forks source link

Receiving deltas from Int64 store fails due to "cannot parse integer from empty string" #7

Closed fschoell closed 1 year ago

fschoell commented 1 year ago

I'm having a max int64 store and I would like to write the store's deltas to a database in my db_out output. This fails with the error:

Error: rpc error: code = Internal desc = step new irr: handler step new: execute modules: running executor "db_out": execute module: execute: execute: maps wasm call: block 2: module "db_out": wasm execution failed: panic in the wasm: "called `Result::unwrap()` on an `Err` value: value  is not a valid representation of an i64\n\nCaused by:\n    cannot parse integer from empty string" at /Users/work/.cargo/registry/src/github.com-1ecc6299db9ec823/substreams-0.5.0/src/store.rs:1279:10

The code failing on the store.rs is:

// We accept &Vec<u8> instead of &[u8] because use internally and makes it easier to chain
fn decode_bytes_to_i64(bytes: &Vec<u8>) -> i64 {
    // FIXME: If we are ready to accept the fact that `bytes` is always valid UTF-8, we could even use
    //        the unsafe `from_utf8_unchecked` version, we would need first to measure the impact and
    //        better understand implication of an invalid UTF-8 &str with `from_str` call.
    let int_as_str =
        std::str::from_utf8(bytes).expect("received bytes expected to be valid UTF-8 string");

    i64::from_str(int_as_str)
        .with_context(|| {
            format!(
                "value {} is not a valid representation of an i64",
                int_as_str
            )
        })
        .unwrap()
}

Some extracted sample code from my substream's lib.rs:

#[substreams::handlers::store]
fn store_max_value(my_input: MyInput, s: StoreMaxInt64) {
    s.max(0, my_input.key, my_input.value);
}

#[substreams::handlers::map]
pub fn db_out(store_max_value: Deltas<DeltaInt64>) -> Result<DatabaseChanges, substreams::errors::Error> {
    for delta in store_max_value.deltas {
        // update the max_trx_block table in case we found a new trx maximum
        if delta.operation == substreams::pb::substreams::store_delta::Operation::Create {
            database_changes.push_change("some_table", &*delta.key, 0, Operation::Create)
                .change("value", (delta.old_value, delta.new_value));
        } else if delta.operation == substreams::pb::substreams::store_delta::Operation::Update {
            database_changes.push_change("some_table", &*delta.key, 0, Operation::Update)
                .change("value", (delta.old_value, delta.new_value));
        }
    }
}

Briefly tested the AddInt64 store which has the same issue. Could it be that the old_value of the first store entry is just empty and thus fails to decode?

maoueh commented 1 year ago

Seems likely indeed, so it's within substreams-rs right, when it receives a delta of the store_max_value right?

fschoell commented 1 year ago

Seems likely indeed, so it's within substreams-rs right, when it receives a delta of the store_max_value right?

Yes

colindickson commented 1 year ago

@fschoell reached out to you on Slack for more information.

colindickson commented 1 year ago

fixed in version 0.5.1. thank you!