Suppose a query result returns a large number of rows:
let rows = try db.prepare(...)
To access the data, I have to loop through each row, and on each row lookup the desired columns by key. This requires a call to Row.get which does a lot of work (quoting the column name, looking it up in a dictionary, checking for ambiguity, etc).
for row in rows {
let x = row.get(Expression<double>("value"))
}
This is a lot of needless work, when each row contains the exact same information.
Alternatively, I should be able to look up the index of the column first, and then simply access it from each row.
Here is a mockup of what this might look like:
let valueIndex = rows.first!.findIndex("value")
for row in rows {
let x: Double = row.get(valueIndex)
}
Motivation
This is killing my performance when decoding a large number of rows (20,000 with 10 columns each).
This suggestion also better matches how the SQLite C api works, and most SQLite libraries (such as Python's standard library).
Possible implementation
I think we can get there with a few small changes:
Refactor Row.get into two functions: Row.get and Row.getIndex.
getIndex will have almost all of the code currently there.
get can then immediately call getIndex.
Make Row.values public.
Alternative
It's possible that the SQLiteEncoder and SQLiteDecoder could modified to decode arrays of codable objects. This could then efficiently do something like described above.
Overview
Suppose a query result returns a large number of rows:
To access the data, I have to loop through each row, and on each row lookup the desired columns by key. This requires a call to
Row.get
which does a lot of work (quoting the column name, looking it up in a dictionary, checking for ambiguity, etc).This is a lot of needless work, when each row contains the exact same information. Alternatively, I should be able to look up the index of the column first, and then simply access it from each row. Here is a mockup of what this might look like:
Motivation
This is killing my performance when decoding a large number of rows (20,000 with 10 columns each).
This suggestion also better matches how the SQLite C api works, and most SQLite libraries (such as Python's standard library).
Possible implementation
I think we can get there with a few small changes:
Refactor
Row.get
into two functions:Row.get
andRow.getIndex
.getIndex
will have almost all of the code currently there.get
can then immediately callgetIndex
.Make
Row.values
public.Alternative
It's possible that the SQLiteEncoder and SQLiteDecoder could modified to decode arrays of codable objects. This could then efficiently do something like described above.