stephencelis / SQLite.swift

A type-safe, Swift-language layer over SQLite3.
MIT License
9.57k stars 1.54k forks source link

Should be able to access columns by index, not just by key. #1133

Closed justinmeiners closed 2 years ago

justinmeiners commented 2 years ago

Overview

Suppose a query result returns a large number of rows:

let rows = try db.prepare(...)

To access the data, I have to loop through each row, and on each row lookup the desired columns by key. This requires a call to Row.get which does a lot of work (quoting the column name, looking it up in a dictionary, checking for ambiguity, etc).

 for row in rows {
     let x = row.get(Expression<double>("value"))
 }

This is a lot of needless work, when each row contains the exact same information. Alternatively, I should be able to look up the index of the column first, and then simply access it from each row. Here is a mockup of what this might look like:

let valueIndex = rows.first!.findIndex("value")

for row in rows {
      let x: Double = row.get(valueIndex)
}

Motivation

This is killing my performance when decoding a large number of rows (20,000 with 10 columns each).

This suggestion also better matches how the SQLite C api works, and most SQLite libraries (such as Python's standard library).

Possible implementation

I think we can get there with a few small changes:

  1. Refactor Row.get into two functions: Row.get and Row.getIndex. getIndex will have almost all of the code currently there. get can then immediately call getIndex.

  2. Make Row.values public.

Alternative

It's possible that the SQLiteEncoder and SQLiteDecoder could modified to decode arrays of codable objects. This could then efficiently do something like described above.

justinmeiners commented 2 years ago

This can be done using a raw query and returning a Statement.