runQuery で using parameter error

Quoted[Action[A]] に対する run ならうまくいくのに

def runAction[A](quotation: Quoted[Action[A]])(
  using Encoder[A], Decoder[A]
): Long =
  val ctx = connect
  import ctx._
  val res = run(quotation)
  ctx.close
  res

Quoted[Query[_]] に対する run はうまくいかない。

def runQuery[A](quotation: Quoted[Query[A]])(
  using Encoder[A], Decoder[A]
): List[A] =
  val ctx = connect
  import ctx._
  val res = run(quotation)
  ctx.close
  res

[error] -- Error: /Users/tomoki.mizogami/src/protoquill-sandbox/src/main/scala/repository/Repository.scala:42:17
[error] 42 |    val res = run(quotation)
[error]    |              ^^^^^^^^^^^^^^
[error]    |No Decoder found for A and it is not a class representing a group of columns
[error]    | This location contains code that was inlined from Repository.scala:42
[error]    | This location contains code that was inlined from Context.scala:120
[error]    | This location contains code that was inlined from Context.scala:110
[error]    | This location contains code that was inlined from JdbcContext.scala:38

また、 Quoted[ActionReturning[A, B]] と Quoted[ActionReturning[A, List[B]]] もうまくいかない。

def runAction[A, B](quotation: Quoted[ActionReturning[A, List[B]]])(
  using Encoder[A], Encoder[B], Decoder[A], Decoder[B]
): List[B] =
  val ctx = connect
  import ctx._
  val res = run(quotation)
  ctx.close
  res

def runAction[A, B](quotation: Quoted[ActionReturning[A, B]])(
  using Encoder[A], Encoder[B], Decoder[A], Decoder[B]
): B =
  val ctx = connect
  import ctx._
  val res = run(quotation)
  ctx.close
  res

以下について疑問

Encoder, Decoder を using パラメータでどこで渡している？

inline def だから using パラメータは展開後のコードにしか出てこないのか。

run メソッドにはいくつかの種類があり、JdbcContext にて定義されている： https://github.com/zio/zio-protoquill/blob/v4.2.0/quill-jdbc/src/main/scala/io/getquill/context/jdbc/JdbcContext.scala#L37-L52

Quoted[Query[T]] に関する run メソッドは List[T] を返す。 InternalApi の runQueryDefault メソッドを実行する。

inline def run[T](inline quoted: Quoted[Query[T]]): List[T] = InternalApi.runQueryDefault(quoted)

ここで、 T は Person や Task といったスキーマモデルの型であり、 Query[T] は問い合わせの SQL を表現するものである。Query[_] については別のところで調査する。

runQueryDefault メソッドは、quotation を受け取って Result[RunQueryResult[T]] を返す。実際には共通の runQuery を呼び出しているだけである。

inline def runQueryDefault[T](inline quoted: Quoted[Query[T]]): Result[RunQueryResult[T]] =
  runQuery(quoted, OuterSelectWrap.Default)

runQuery メソッドは、quotation と OuterSelectWrap を受け取って Result[RunQueryResult[T]] を返す。

JdbcContext における Result[T] と RunQueryResult[T] 型は以下で定義される。すなわち、 runQueryDefault の返り値は List[T] である。

  override type Result[T] = T
  override type RunQueryResult[T] = List[T]

OuterSelectWrap には OuterSelectWrap.Default を渡している。

https://github.com/zio/zio-protoquill/blob/v4.2.0/quill-sql/src/main/scala/io/getquill/context/Context.scala#L115-L121

OuterSelectWrap ？ OuterSelectWrap は enum である。ただ、今は必要ないらしい。Quill 側にあるやつなのかな、一旦無視しよう。 https://github.com/zio/zio-protoquill/blob/v4.2.0/quill-sql/src/main/scala/io/getquill/OuterSelect.scala#L11

/**
 * TODO Not needed now since elabration does not do OntoAst?
 */
enum OuterSelectWrap:
  case Always
  case Never
  case Default

inline def runQuery[T](inline quoted: Quoted[Query[T]], inline wrap: OuterSelectWrap): Result[RunQueryResult[T]] = {
  val ca = make.op[Nothing, T, Result[RunQueryResult[T]]] { arg =>
    val simpleExt = arg.extractor.requireSimple()
    self.executeQuery(arg.sql, arg.prepare.head, simpleExt.extract)(arg.executionInfo, _summonRunner())
  }
  QueryExecution.apply(ca)(quoted, None, wrap)
}

make: ContextOperation.Factory
- Idiom, NamingStrategy は、JdbcContext を作るときに渡している設定クラス。
- PrepareRow, ResultRow, Session はよくわからん。self.type は Context の型。よくわからんけど、全部 Context から渡している。
- Context は ProtoContextSecundus と EncodingDsl, Closeable を継承している。この EncodingDsl 内で PrepareRow, ResultRow, Session を定義している。
- JdbcContext では、 JdbcContextTypes にて具象型が定義されている。

private lazy val make = ContextOperation.Factory[Dialect, Naming, PrepareRow, ResultRow, Session, self.type](self.idiom, self.naming)

trait Context[+Dialect <: Idiom, +Naming <: NamingStrategy]
    extends ProtoContextSecundus[Dialect, Naming] with EncodingDsl with Closeable:

trait EncodingDsl extends LowPriorityImplicits { self => // extends LowPriorityImplicits
  type PrepareRow
  type ResultRow
  type Session

trait JdbcContextTypes[+Dialect <: SqlIdiom, +Naming <: NamingStrategy] extends Context[Dialect, Naming]:
...
  // Dotty doesn't like that this is defined in both Encoders and Decoders.
  // Makes us define it here in order to resolve the conflict.
  type Index = Int
  type PrepareRow = PreparedStatement
  type ResultRow = ResultSet
  type Session = Connection
  type Runner = Unit

make.op[Nothing, T, Result[RunQueryResult[T]]
- Factory の op メソッドは、 ContentOperation クラスのインスタンスを生成するメソッドである。
- ContextOperation クラスは、Idiom と NamingStorategy の設定と execute を引数として持つ。execute は、Argument を受け取って Result を返す関数である。
- Argument は、sql, prepare, extractor, executionInfo, fetchSize をフィールドに持つ。
- JdbcContext の Query 実行に関して、Res 型は List[T] である。

def op[I, T, Res] =
  ContextOperation[I, T, Nothing, D, N, PrepareRow, ResultRow, Session, Ctx, Res](idiom, naming)

case class ContextOperation[I, T, A <: QAC[I, _] with Action[I], D <: Idiom, N <: NamingStrategy, PrepareRow, ResultRow, Session, Ctx <: Context[_, _], Res](val idiom: D, val naming: N)(
    val execute: (ContextOperation.Argument[I, T, A, D, N, PrepareRow, ResultRow, Session, Ctx, Res]) => Res
)

case class Argument[I, T, A <: QAC[I, _] with Action[I], D <: Idiom, N <: NamingStrategy, PrepareRow, ResultRow, Session, Ctx <: Context[_, _], Res](
    sql: String,
    prepare: Array[(PrepareRow, Session) => (List[Any], PrepareRow)],
    extractor: Extraction[ResultRow, Session, T],
    executionInfo: ExecutionInfo,
    fetchSize: Option[Int]
)

arg.extractor
- Extraction[ResultRow, Session, T] 型である。
- Extraction が何を意味しているのかはよくわからん。
- Extraction の具象クラスによれば、返り値がそのままになるか、Returning になるか Nothing になるか表しているのかな。

object Extraction:
  case class Simple[ResultRow, Session, T](extract: (ResultRow, Session) => T) extends Extraction[ResultRow, Session, T]
  case class Returning[ResultRow, Session, T](extract: (ResultRow, Session) => T, returningBehavior: ReturnAction) extends Extraction[ResultRow, Session, T]
  case object None extends Extraction[Any, Any, Nothing]

arg.extractor.requireSimple()
- requireSimple は、Extraction が Simple であればそのまま返し、それ以外の場合は例外を吐く。
self.executeQuery(arg.sql, arg.prepare.head, simpleExt.extract)(arg.executionInfo, _summonRunner())
- executeQuery は ProtoContextSecundus にてシグネチャが定義されている。その実装は JdbcContextVerbExecute にて定義されている。

def executeQuery[T](sql: String, prepare: Prepare, extractor: Extractor[T])(executionInfo: ExecutionInfo, rn: Runner): Result[RunQueryResult[T]]

// Not overridden in JdbcRunContext in Scala2-Quill because this method is not defined in the context
override def executeQuery[T](sql: String, prepare: Prepare = identityPrepare, extractor: Extractor[T] = identityExtractor)(info: ExecutionInfo, dc: Runner): Result[List[T]] =
  withConnectionWrapped { conn =>
    val (params, ps) = prepare(conn.prepareStatement(sql), conn)
    logger.logQuery(sql, params)
    val rs = ps.executeQuery()
    extractResult(rs, conn, extractor)
  }

QueryExecution.apply(ca)(quoted, None, wrap)

Jdbc における ps.executeQuery は、 java.sql.PreparedStatement のメソッド。

/**
 * Executes the SQL query in this <code>PreparedStatement</code> object
 * and returns the <code>ResultSet</code> object generated by the query.
 *
 * @return a <code>ResultSet</code> object that contains the data produced by the
 *         query; never <code>null</code>
 * @exception SQLException if a database access error occurs;
 * this method is called on a closed  <code>PreparedStatement</code> or the SQL
 *            statement does not return a <code>ResultSet</code> object
 * @throws SQLTimeoutException when the driver has determined that the
 * timeout value that was specified by the {@code setQueryTimeout}
 * method has been exceeded and has at least attempted to cancel
 * the currently running {@code Statement}
 */
ResultSet executeQuery() throws SQLException;

ここで実際に SQL を実行するのか？

あ、でも、↓では実行方法を定義しているだけで、 ca はただの ContextOperation なのか。ContextOperation の execute を評価するタイミングはおそらく QueryExecution.apply の中。

val ca = make.op[Nothing, T, Result[RunQueryResult[T]]] { arg =>
  val simpleExt = arg.extractor.requireSimple()
  self.executeQuery(arg.sql, arg.prepare.head, simpleExt.extract)(arg.executionInfo, _summonRunner())
}

QueryExecution

QueryExecution.apply

QueryExecution.apply(ca)(quoted, None, wrap)

inline def apply[
    I,
    T,
    DecodeT,
    ResultRow,
    PrepareRow,
    Session,
    D <: Idiom,
    N <: NamingStrategy,
    Ctx <: Context[_, _],
    Res
](
  ctx: ContextOperation[I, T, Nothing, D, N, PrepareRow, ResultRow, Session, Ctx, Res]
)(
  inline quotedOp: Quoted[QAC[_, _]],
  fetchSize: Option[Int],
  inline wrap: OuterSelectWrap = OuterSelectWrap.Default
) = ${ applyImpl('quotedOp, 'ctx, 'fetchSize, 'wrap) }

apply の処理は applyImpl にマクロで書かれている。

def applyImpl[
    I: Type,
    T: Type,
    DecodeT: Type,
    ResultRow: Type,
    PrepareRow: Type,
    Session: Type,
    D <: Idiom: Type,
    N <: NamingStrategy: Type,
    Ctx <: Context[_, _]: Type,
    Res: Type
](
    quotedOp: Expr[Quoted[QAC[_, _]]],
    ctx: Expr[ContextOperation[I, T, Nothing, D, N, PrepareRow, ResultRow, Session, Ctx, Res]],
    fetchSize: Expr[Option[Int]],
    wrap: Expr[OuterSelectWrap]
)(using qctx: Quotes): Expr[Res] =
  new RunQuery[I, T, ResultRow, PrepareRow, Session, D, N, Ctx, Res](quotedOp, ctx, fetchSize, wrap).apply()

RunQuery#apply

def apply() =
  // Since QAC type doesn't have the needed info (i.e. it's parameters are existential) hence
  // it cannot be checked if they are nothing etc... so instead we need to check the type
  // on the actual quoted term.
  quotedOp.asTerm.tpe.asType match
    // Query has this shape
    case '[Quoted[QAC[Nothing, _]]] => applyQuery(quotedOp)
    // Insert / Delete / Update have this shape
    case '[Quoted[QAC[_, Nothing]]] => applyAction(quotedOp)
    // Insert Returning, ReturningMany, etc... has this shape
    case '[Quoted[QAC[_, _]]] =>
      if (!(TypeRepr.of[T] =:= TypeRepr.of[Any]))
        applyActionReturning(quotedOp) // ReturningAction is also a subtype of Action so check it before Action
      else
        // In certain situations (i.e. if a user does infix"stuff".as[Action[Stuff]] something will be directly specified
        // as an Action[T] without there being a `& QAC[T, Nothing]` as part of the type. In that case, the `ModificationEntity`
        // will just be `Any`. We need to manually detect that case since it requires no return type)
        applyAction(quotedOp)
    case _ =>
      report.throwError(s"Could not match type type of the quoted operation: ${io.getquill.util.Format.Type(QAC)}")

Query の場合は applyQuery

/**
 * Summon all needed components and run executeQuery method
 * (Experiment with catching `StaticTranslationMacro.apply` errors since they usually happen
 * because some upstream construct has done a reportError so we do not want to do another one.
 * I.e. if we do another returnError here it will override that one which is not needed.
 * if this seems to work well, make the same change to other apply___ methods here.
 * )
 */
def applyQuery(quoted: Expr[Quoted[QAC[_, _]]]): Expr[Res] =
  val topLevelQuat = QuatMaking.ofType[T]
  summonQueryMetaTypeIfExists[T] match
    // Can we get a QueryMeta? Run that pipeline if we can
    case Some(queryMeta) =>
      queryMeta match { case '[rawT] => runWithQueryMeta[rawT](quoted) }
    case None =>
      Try(StaticTranslationMacro[D, N](quoted, queryElaborationBehavior, topLevelQuat)) match
        case scala.util.Failure(e) =>
          import CommonExtensions.Throwable._
          val msg = s"Query splicing failed due to error: ${e.stackTraceToString}"
          // TODO When a trace logger is found instrument this
          // println(s"[InternalError] ${msg}")
          // Return a throw if static translation failed. This typically results from a higher-level returnError that has already returned
          // if we do another returnError here it will override that one which is not needed.
          report.throwError(msg)
        // Otherwise the regular pipeline
        case scala.util.Success(Some(staticState)) =>
          executeStatic[T](staticState, identityConverter, ExtractBehavior.Extract, topLevelQuat) // Yes we can, do it!
        case scala.util.Success(None) =>
          executeDynamic(quoted, identityConverter, ExtractBehavior.Extract, queryElaborationBehavior, topLevelQuat) // No we can't. Do dynamic

executeDynamic にて、Decoder を summon してそう。

実行結果を List[T] にするタイミングで No Decoder found になってるっぽいから、 ResultSet から List[T] を作るタイミングだったりする？

ResultSet を Result[List[T]] に変換しているところは JdbcContext#executeQuery の extractResult。

 private[getquill] final def extractResult[T](rs: ResultSet, conn: Connection, extractor: Extractor[T]): List[T] =
   ResultSetExtractor(rs, conn, extractor)

ResultSetExtractor

extractor ←こいつか。

extractor: (ResultSet, Connection) => T

こいつは、 args.extractor.requireSimple() で取得した extractor のこと。Argument の生成時に extractor は決まってくるのかな。 Argument はいつ生成される？

executeDynamic 内のここで作られてそう。

ctx.execute(ContextOperation.Argument(queryString, Array(prepare), extractor, ExecutionInfo(ExecutionType.Dynamic, executionAst, topLevelQuat), fetchSize))

extractor ?

val (queryString, outputAst, sortedLifts, extractor, _) =
  PrepareDynamicExecution[I, T, RawT, D, N, PrepareRow, ResultRow, Session](quoted, rawExtractor, ctx.idiom, ctx.naming, elaborationBehavior, topLevelQuat)

PrepareDynamicExecution で作っている。長い。。。

val extractor = (rawExtractor, returningActionOpt) match
  case (Extraction.Simple(extract), Some(returningAction)) => Extraction.Returning(extract, returningAction)
  case (Extraction.Simple(_), None)                        => rawExtractor
  case (Extraction.None, None)                             => rawExtractor
  case (extractor, returningAction)                        => throw new IllegalArgumentException(s"Invalid state. Cannot have ${extractor} with a returning action ${returningAction}")

rawExtractor ? は↓

val extractor: Expr[io.getquill.context.Extraction[ResultRow, Session, T]] = MakeExtractor[ResultRow, Session, T, RawT].dynamic(converter, extract)

MakeExtractor.dynamic

ExtractBehavior.Extract

val extractor = makeExtractorFrom(converter)

converter is

// Simple ID function that we use in a couple of places
def identityConverter[T: Type](using Quotes) = '{ (t: T) => t }

つまり、extractor の実装は↓。この時点で Decoder[T] が必要になる。

def makeExtractorFrom(contramap: Expr[RawT => T])(using Quotes) =
  val decoder = makeDecoder[ResultRow, Session, RawT]()
  '{ (r: ResultRow, s: Session) => $contramap.apply(${ decoder }.apply(0, r, s)) }

Summon decoder

https://github.com/zio/zio-protoquill/blob/v4.2.0/quill-sql/src/main/scala/io/getquill/context/QueryExecution.scala#L93

/** Summon decoder for a given Type and Row type (ResultRow) */
def summonDecoderOrThrow[ResultRow: Type, Session: Type, DecoderT: Type]()(using Quotes): Expr[GenericDecoder[ResultRow, Session, DecoderT, DecodingType]] =
  import quotes.reflect.{Try => _, _}
  // First try summoning a specific encoder, if that doesn't work, use the generic one.
  // Note that we could do Expr.summon[GenericDecoder[..., DecodingType.Generic]] to summon it
  // but if we do that an error is thrown via report.throwError during summoning then it would just be not summoned and the
  // and no error would be returned to the user. Therefore it is better to just invoke the method here.
  Expr.summon[GenericDecoder[ResultRow, Session, DecoderT, DecodingType.Specific]] match
    case Some(decoder) => decoder
    case None =>
      GenericDecoder.summon[DecoderT, ResultRow, Session]

taretmch / protoquill-sandbox

runQuery で using parameter error #1

QueryExecution

マクロ

Summon decoder