Open ex0ns opened 1 year ago
QueryExecution.scala => processLifts
function has bugs to handle the same quote
used more than one time.
I wanted to dig a bit deeper in order to solve this issue but I could not find the time to, our solution (for those interested) is pretty pretty ugly and might failed in other setup is to manually update the UUID so they are unique in the final query:
//> using scala "3.3"
//> using dep io.getquill::quill-sql::4.8.0
import io.getquill.*
import io.getquill.ast.*
@main
def main =
case class Custom(name: String)
val names = Seq("a", "b")
val ctx = new SqlMirrorContext(MirrorSqlDialect, Literal)
import ctx.*
def createNameFilters(names: Seq[String])(p: Quoted[Custom]) =
val nameFilters = names.map(name => quote(p.name == lift(name)))
// val nameFilters = names.map(name => quote(p.name.like(lift(name))))
val uuids = names.map(_ => java.util.UUID.randomUUID.toString)
// this part should be way more robust, a better way of doing that would be to:
// 1. retrieve all the current `lifts` (filters.map(_.lifts))
// 2. keep a map of old uuid => new uuid
// 3. Traverse the AST and replace old uuid by new uuid
val filters: Seq[Quoted[Boolean]] = nameFilters.zip(uuids).map: (filter, uuid) =>
filter.copy(
ast = filter.ast match
case infix @ Infix(_, _ @head :: (last: ScalarTag) :: Nil , _, _, _) =>
infix.copy(params = head :: last.copy(uid = uuid) :: Nil)
case bop @ BinaryOperation(_, _ , op: ScalarTag) =>
bop.copy(b = op.copy(uid = uuid))
case other => other
,
lifts = filter.lifts.map:
case lift: EagerPlanter[_, _, _] => lift.copy(uid = uuid)
case other => other
)
filters.reduce((l, r) => quote(l || r))
val filters = createNameFilters(names)
val query = dynamicQuery[Custom].filter(p => filters(p))
println(ctx.translate(query))
// # SELECT v0.name FROM Custom v0 WHERE v0.name = 'a' OR v0.name = 'b'
I would really not recommend using this code, but this might give some hints on the fix
Okay, I've looked into this. The problem here is that we cannot assume that all lifts with a common UUID (i.e. the UUID assigned at compile-time) will occur in order. This will even be a problem if the reduction-step in the above example is filtered or re-ordered in some way (I can try to come up with examples of this in the coming days).
What is really required here is a secondary-uuid that lifts produce at runtime. This needs to be added to the Planter class as a new variable runtimeUid:UUID
which will be carried around on the base PlanterExpr as a Expr[UUID]
. That needs to be added via the liftValue
in the LiftMacro:
'{ EagerPlanter($valueEntity, $encoder, ${ Expr(uuid) }) }
to:
'{ EagerPlanter($valueEntity, $encoder, ${ Expr(uuid) }, java.util.UUID.randomUUID().toString) }
We will propagate these things from the EagerPlanter implementations to the EagerPlanterExpr implementations (the LazyPlanter should do the same kind of thing possibly but we can leave it alone for now) and eventually the additionalLifts
argument of the PrepareDynamicExecution
invocation in QueryExecutionBatchDynamic
.
The only major problem with this is how to pass around the runtime-uuid on the ScalarTags. Right now ScalarTag assumes the UID is always know at compile-time which clearly is not the case if we make a change like this. Probably the lifter/unlifter needs to be modified to be more like the Scala 2 lifter/unlifter which only serializes Quats, not the whole tree. ScalarTag will need a runtimeUid:Expr[String] which it gets from the parser i.e. this thing:
case PlanterExpr.UprootableUnquote(expr) =>
ScalarTag(expr.uid, expr.runtimeUid /*add this*/, Source.Parser)
The lifter needs to splice this value directly into a SplicedScalarTag object: from:
given liftScalarTag: LiftAstSerialize[ScalarTag] with {
def typeTag = TType.of[ScalarTag]
def lift = {
case ScalarTag(uid: String, source) => '{ ScalarTag(${ uid.expr }, ${ source.expr }) }
}
}
to
given liftScalarTag: LiftAstSerialize[ScalarTag] with {
def typeTag = TType.of[SplicedScalarTag]
def lift = {
case ScalarTag(uid: String, runtimeUid, source) => '{ SplicedScalarTag(${ uid.expr }, $runtimeUid, ${ source.expr }) }
}
}
A similar change needs to happen to the unlifter.
Let me think about this issue some more and I'll flush the rest out.
We have a part of the code that apply a filter on a dynamic query, for that we use a map that looks like that
But only the last element of the
nameFilter
is kept (it is actually duplicated). In the reproducer I have remove thelike
part as the behavior is the same with==
as well.I don't have enough knowledge about quill's internal but it seems that the variable that is capture is always the latest in the list, but I can not determine if this is coming from the map or the reduce
Version: 4.6.0.1 Module: sql Database: all
Expected behavior
This should output the following query:
Actual behavior
Steps to reproduce the behavior
Workaround
None found yet, but maybe an infix operator could fix that.
More infos
When printing the list of generated queries, I get the following:
And as far as I can tell, the
a
andb
seems to be correct here however (I don't know if this is relevant or not) theScalarTag 74f64d15-44dc-4bc6-a7dc-ac6eb8b47da3 is the same in both cases
My guess is that as we have a single
lift
(as opposed to liftQuery) for the collection, we are considering the last lift @getquill/maintainers