manticoresoftware / manticoresearch

Easy to use open source fast database for search | Good alternative to Elasticsearch now | Drop-in replacement for E in the ELK soon
https://manticoresearch.com
GNU General Public License v3.0
9.02k stars 504 forks source link

Feature Request: MVA add and remove operations #1823

Open etcd opened 8 months ago

etcd commented 8 months ago

Is your feature request related to a problem? Please describe. According to the documentation, MVAs are stored internally as a sorted set. However, they don't support basic set update operations such as add and remove, and these would be very nice to have.

Describe the solution you'd like It would be nice for MVAs to support basic set update operations like add and remove.

Describe alternatives you've considered Making any update to an MVA value requires replacing the entire value. For instance, given the following table and initial data:

CREATE TABLE product(tags multi);
INSERT INTO product VALUES (1, (4,2,1,3));

The only way to remove a value from the MVA (e.g., 4) is by replacing the entire MVA:

UPDATE product SET tags=(2,1,3) WHERE id=1;

In this toy example with only a few values, it's not too bad. However, this is very unwieldy and inefficient when working with large MVAs. For example, to remove a value from an MVA, you currently have to SELECT the MVA value, then find the value to remove, then perform an UPDATE query. That's two queries and a lot of bandwidth - not to mention, if MVAs grow to be very large (say 1000 multi64 values = 8KB), you will also run into a practical limit to the size of the UPDATE query.

sanikolaev commented 4 months ago

Some ideas on the syntax:

etcd commented 4 months ago

I like that. Not sure if this reduces implementation workload drastically, but if the first two exist, the third doesn't seem necessary (or conversely, if the third exists, no need for first two).