Handling output from `FT.SEARCH` in JSON

[HerrMuellerluedenscheid]

Hi,

I have an index usernames of JSON objects. When running FT.SEARCH I get results, where the first returned item is the number of found entries and the following are key-value pairs. I did not manage to extract the data and get the underlying JSON objects. What I have achieved so far is the following which successfully iters over the found key-value pairs of the index query:

let result: Value = redis::cmd("FT.SEARCH").arg("idx:username").arg(username).query_async(&mut self.connection).await.unwrap();
for (_, v) in result.as_sequence()     // We ignore the first element of the tuple which contains the Path ($).
                .unwrap()
                .iter()
                .skip(1)      // We skip the first entry which contains the number of found items
                .into_iter()
                .tuples() {
            println!("{v:?}");
        }

Which prints:

bulk(string-data('"$"'), string-data('"{\"user_id\":\"9511c06b-4f59-4fca-8ac5-fa544d7c1cdf\",\"username\":\"Peter\",\"password_hash\":\"$argon2id$v=19$m=4096,t=192,p=8$fpL+GDtUBZ1MMzgppZ3VUaz11w+rBqTr3umNY1kDxWw$TZLCDdX4ZFAMykPWodwnwdDJw6lS/QyG9SaGK31quSI\"}"'))
bulk(string-data('"$"'), string-data('"{\"user_id\":\"09f53c2e-421a-4b88-9aa4-5084e4c3111f\",\"username\":\"Peter\",\"password_hash\":\"$argon2id$v=19$m=4096,t=192,p=8$02yKNoQzWk1NKw5t7iHh0EpEQqWOPfK9U6h42D3lWcI$E0dp+o5tuJqzMDZ7v8F+nOB0sSEL7l+RGUOzHS1MRw8\"}"'))
bulk(string-data('"$"'), string-data('"{\"user_id\":\"9fd64f6f-bb5c-4a99-868a-5280191d1880\",\"username\":\"Peter\",\"password_hash\":\"$argon2id$v=19$m=4096,t=192,p=8$9zKdgiZZzL0P9Kp5dTq4MgtqTG5RN4q7SVAw3dlN5yc$fGNscohzAQlpjEim6KXx0yrBntGGTl1HfJGnR1+sUdM\"}"'))

How can I unpack the result to finally deserialize the string-data of each iteration?

Thanks in advance!

[HerrMuellerluedenscheid]

I made a little tiny step forward. I have used the redis-derive crate for my User struct and tried the from_redis_value on each element:

        let result: Value = redis::cmd("FT.SEARCH").arg("idx:username").arg(username).query_async(&mut self.connection).await.unwrap();
        for (_, v) in result.as_sequence()
                .unwrap()
                .iter()
                .skip(1)
                .into_iter()
                .tuples() {
            println!("{v:?}");
            User::from_redis_value(v).expect("TODO: panic message");
        }

unfortunately, that panics with:

he data returned from the redis database was not in the bulk data format or the length of the bulk data is not devisable by two.

EDIT: just discovered that this is an error message defined in the redis-derive crate.

[wojciechbator]

I think I have a hint for you. To parse this output do these things: 1: result should be of type Vec<Value> . For brevity, I assume that Value is equal to redis::Value from this lib. 2: Parsing part:

// assuming this command returns results as you mentioned
let result: Vec<Value> = redis::cmd("FT.SEARCH").arg("idx:username").arg(username).query_async(&mut self.connection).await?;
// result map of key:value from FT.SEARCH
let map: HashMap<String, YourDeserializeTarget> = HashMap::new();
let mut i = result.into_iter();
// skip first, it's total results, probably useful for pagination, if you have default limit of 10 and more results
let _ = i.next();

while let (Some(key), Some(value)) = (i.next(), i.next()) {
    // Value::Data is a key string and Value::Bulk is a value from FT.SEARCH command
    if let (Value::Data(k), Value::Bulk(v)) = (&key, &value) {
        let key_str = String::from_utf8(k.to_vec())?;
        if let Value::Data(json_data) = &v[1] {
            let json_str = String::from_utf8(json_data.clone())?;
            let value: YourDeserializeTarget = serde_json::from_str(&json_str)?;
            map.insert(key_str, value);
        }
    }
}

This can retrieve FT.SEARCH as a map of keys->values. Make sure you're providing a limit if there are more than 10 results of the FT.SEARCH.

PS: If I were you, I'd use a query syntax in command such as:

let result: Vec<Value> = redis::cmd("FT.SEARCH")
            .arg("idx")
            .arg(format!("@username:({})", username))
            .query_async(&mut connection)
            .await?;

This way, if username is a TEXT. If you defined idx with username as a TAG, change curly braces () to {} braces. Tested with dependencies:

  redis = {  version = "0.25.2", features = ["tokio-comp", "aio"] }
  redis-macros = "0.2.1"

@mitsuhiko redisearch could use more examples. This module is totally usable in redis-rs, but it's in a DYI state right now. Consider attaching this example somewhere for users. BTW - great job with the lib.

[heartforit]

Hi,

thank you so much for providing that solution. I was fighting the type system for hours, because IDE support for rust is still very frustrating at some point.

But eventually this did the trick for me:

let result: Vec<Value> = redis::cmd("FT.SEARCH")
        .arg(&index)
        .arg(searchArgs)
        .arg("LIMIT")
        .arg("0") // start
        .arg("1") // count
        .query(&mut con).unwrap();