tonbo-io / tonbo

A portable embedded database using Arrow.
https://tonbo.io
Apache License 2.0
807 stars 46 forks source link

How to add a Float32Array data type? #162

Open sxhxliang opened 2 months ago

sxhxliang commented 2 months ago

Question

How to add a Float32Array data type? eg:

// insert with owned value
db.insert(User {
    name: "Alice".into(),
    email: Some("alice@gmail.com".into()),
    age: 22,
    bytes: Bytes::from(vec![0, 1, 2]),
    float32array: vec![0.1, 1.1, 2.0],
})
.await
.unwrap();

Before asking a question, make sure you have:

ethe commented 2 months ago

This is not supported yet, and we are working on aligning Tonbo with Arrow data types. Do you have any good suggestions on this topic? I would appreciate it if you would explain scenarios your are facing, and why they rely on these types.

sxhxliang commented 2 months ago

For example, vector search requires storing the semantic vector of the text.

ethe commented 2 weeks ago

refer to #211