`Parameter error` when filtering on large i64 values

[bmuddha]

Hello, I've been fighting with this issue for some time now, and cannot seem to understand what's the root of the problem. Let's imagine we have the following code:

use aerospike::{
    expressions::{eq, int_bin, int_val},
    Bin, Bins, Client, ClientPolicy, Key, QueryPolicy, Statement, Value, WritePolicy,
};

fn main() {
    let c = Client::new(&ClientPolicy::default(), &"172.17.0.1:3000").unwrap();
    let pkey = Pubkey::new("Feature111111111111111111111111111111111111");
    let acc = Account {
        owner: pkey.clone(),
        data: vec![1; 32],
    };

    let key = Key::new("test", "set", Value::Int(1)).unwrap();
    c.put(&WritePolicy::default(), &key, &acc.into_bins())
        .unwrap();
    let mut qpolicy = QueryPolicy::new();
    let filter = eq(int_bin("oidx".into()), int_val(pkey.hash() as i64));
    qpolicy.filter_expression.replace(filter);
    let stmt = Statement::new("test", "set", Bins::All);
    let set = c.query(&qpolicy, stmt).unwrap();
    for s in &*set {
        if let Err(e) = s {
            eprintln!("Error querying aerospike: {}", e);
        }
    }
}

struct Account {
    owner: Pubkey,
    data: Vec<u8>,
}

impl Account {
    fn into_bins(&self) -> [Bin<'static>; 3] {
        let owner = Bin::new("owner", Value::from(&self.owner));
        let data = Bin::new("data", Value::from(&self.data));
        let oidx = Bin::new("oidx", Value::from(self.owner.hash() as i64));

        [owner, data, oidx]
    }
}

#[derive(Clone)]
struct Pubkey(Vec<u8>);

impl From<&Pubkey> for Value {
    fn from(key: &Pubkey) -> Self {
        key.0.clone().into()
    }
}

impl Pubkey {
    fn hash(&self) -> u64 {
        use std::collections::hash_map::DefaultHasher;
        use std::hash::{Hash, Hasher};
        let mut hasher = DefaultHasher::new();
        self.0.hash(&mut hasher);
        hasher.finish()
    }

    fn new(s: &str) -> Self {
        let vec = bs58::decode(s).into_vec().unwrap();
        Self(vec)
    }
}

Nothing fancy, though a little lengthy, so the issue arises when one tries to query this set using filter expressions:eq filters on oidx bin, which stores Value::Int, library just reports that there was a Parameter error while on server one can observe a warning

WARNING (exp): (exp.c:1282) build_count_sz - invalid op_code 105 size 0
WARNING (scan): (scan.c:706) basic scan job failed filter processing

What makes this issue interesting, is that it only happens with large positive numbers, yes, with negatives it doesn't happen, no matter how large the number is (e.g. hash of Sysvar1111111111111111111111111111111111111), also it doesn't happen with small numbers (not sure what the range is, but it's probably pretty large). I would've thought that this might be a server issue (aerospike works this way may be), but if you make this query via aql

SELECT * FROM test.set WHERE oidx = 327397455200479690

Then everything works out fine. Additionally if one were to use Statement::add_filter to add the same filter, then it works with any valid i64 values, without causing Parameter error issue.

I'll try to provide any additional context if necessary, thanks.

[kportertx]

Could you set the server's "exp" logging context to detail and run the test again. The server will print a couple new logs, one of which will provide a hex dump of what the client sent the server.

[bmuddha]

Feb 22 2022 16:08:34 GMT: DEBUG (exp): (exp.c:776) as_exp_filter_build - msg_field_sz 20 msg-dump
000000: 93 01 93 51 02 a4 6f 69 64 78 ce 04 8b 26 40 39 ...Q..oidx...&@9
000010: 97 69 ca                                        .i.
Feb 22 2022 16:08:34 GMT: WARNING (exp): (exp.c:1282) build_count_sz - invalid op_code 105 size 0
Feb 22 2022 16:08:34 GMT: WARNING (scan): (scan.c:706) basic scan job failed filter processing

Well, it's kinda hard to make sense of it, but here's an extra log entry with hex dump

[xorphox]

The msgpack breakdown:

93
  01               // (1)eq
  93
    51             // (81)bin
    02
    a4 6f 69 64 78 // string “oidx”
  ce 04 8b 26 40   // uint32
  39               // start of extra stuff (not allowed)
  97
    69             // 105, an invalid op code
    ca  

The wire instruction is a lisp like msgpack, where OPs are encoded as msgpack lists. You can see the first byte says it's a list of 3 elements (0x93). There's the EQ op, which has 2 parameters, the first of which is the BIN op which has 2 parameters (int type, string name). The 2nd parameter of EQ is an integer, and 0xCE means a uint32

https://github.com/msgpack/msgpack/blob/master/spec.md

That should be the end of the instruction as all parameters have been specified, but we have 4 extra bytes which is confusing the sizer on the server. At this point the server is trying to figure out how much memory the compiled code will take up and it ran into OP code 105 in the "extra" part.

Perhaps the uint32 was meant to be a uint64? In which case, it should be 0xCF.

[jonas32]

You are probably right. Look at that: https://github.com/aerospike/aerospike-client-rust/blob/master/src/msgpack/encoder.rs#L344

[khaf]

Fixed in v1.3.0