quoll
commented
3 years ago Query planning is based on the current contents of the database, which means that AOT can't be done on the plan. If the user needs a particular plan, then an argument can be passed in to skip the planning stage.
Queries are provided as either a string, sequence or a map. If it's a string, then that needs parsing. If it's a sequence, then it needs to be split up into the clauses (:find, :in, :where, etc). That's the internal representation. I'm not sure how much AOT can be done after that.
This may be why simple queries in Asami are happening much faster than in Datomic.
Operations to do a simple lookup via {:find ?v :where [:my/entity :value ?v]}
- parse the query. This is a call to
map?and the original data is returned - destructure the map into
:findand:where - attach the default graph to the argument provided in the call to
q - call the planner. This does a count on the
:whereclause, and finds a single vector, so this is returned. - resolve the first element of the planned query:
- this is an index lookup, so look up the index
- project the results (take the results, which are a single value of
?vfor each row, and map each row to a single value of?v. There may be room to optimize here)
It's possible to have some of this done ahead of time, but there's not a lot of scope.
vvvvalvalval
Problem: Clojure Datalog engines tend to have parsing and interpretation overhead, which is frequently significant, and I suspect Asami is no exception.
For example, a Datomic Datalog query will typically have a 'startup time' of 100µs, even if it does nothing more than an index lookup that would take 1µs when done manually.
Since there's already query planning, it would be very nice to be able to compile queries AOT (potentially with partial arguments) into a function.
evalis likely not needed - planning ahead-of-time into a dynamic composition of functions is probably quite enough to trim most of the overhead.