Introduce `in` predicate to check against a set of values

maydanw commented 3 years ago

Problem to Solve

Concrete example:
Given a list of 1000s of document entities ids retrieved from a search in ES we want to check if a user has access to each of them and filter them. The current options are either:

Running 1000s of independent queries
Build a query with 1000s of independent OR statements

Both are inefficient

Current Workaround

Running 1000s of independent queries
Build a query with 1000s of independent OR statements

Proposed Solution

Add "in" (or "isIn") construct to the syntax
For example (in python):

query = "match $d isa document, has objID in ["123", "1122", "333AAA", "6A5B"]; ..."

Additional Information

Beyond improving the ease of usage, I (@maydanw) believe this can be optimized due to:

Server caching - from my understanding lots of the reasoner caching is only inside the tx,
Same server snapshot
It can have better server CPUs utilization
Less network communication
Validating the predicate value in the list can be hashed instead of searched for each.

flyingsilverfin commented 1 year ago

This might naturally be incldued in the future when we include vectors or sets of values natively in TypeDB?

flyingsilverfin commented 1 year ago

vaticle / typeql