fluree / core

Fluree releases and public bug reports
0 stars 0 forks source link

`limit` Appears to Non-Lazily Consume Index Data #42

Open aaj3f opened 8 months ago

aaj3f commented 8 months ago

Description

Working w/ a large db (~70mb of data just in the ex:User class alone), the following query with a limit of 10 takes minutes to attempt to execute then hits a Java heap error

{
    "from": "fabric/db",
    "where": [
        ["?user", "type", "ex:User"]
    ],
    "select": {"?user": ["*"]},
    "limit": 10
}

NOTE: I looked for tests in db on how limit was syntactically managed, but there only appear to be tests for SPARQL and SQL, not FQL. I did try things like "opts": { "limit": 10 } etc, but saw the same behavior

I suppose the expected behavior here would be a lazier approach to index retrieval when a limit is supplied, so that the where-clause execution of index-retrieval won't continue once a sufficient result set has been populated (i.e. limit: 10 and 10 results)

UPDATE: I just reduced the ex:User transacted data down to around 7mb of transacted users so that I'm actually getting query responses. What I'm finding is that I can't get limits on results at all. Either with a top-level "limit": 10 or "opts": { "limit": 10 }, I'm getting 7mb responses of all users that exist