Closed Antar1011 closed 8 years ago
In progress on the integration-test
branch
The species lookup part is done as of e1a95dda60985bba4a7c0dd3c72bb1d5cd48f0a4
Note that if I didn't want to have to rewrite all my unit tests, I could always keep the tuples and then zip with the table columns before insert, but it's not like there's any reason to do that aside from laziness, and this way will result in much clearer tests
Second part is done with ba37f5ccd4e4d081c293c98bbbd94e16ae75a7d7
Currently the integration test throws the following error:
(it goes on for quite some time, but it's referring to the giant CASE clause that is the species lookup)
This is a known limitation with SQLite where one is limited to 999 "
?
" variables. Ref: https://www.sqlite.org/limits.htmlUpon further investigation, I see that the insert expressions are going to have issues as well for any large-scale operations (though this can be mitigated by setting
batch_size
really low).So the solution has two elements:
CASE
expression (there's an alternative where it only combined formes and handles weird cases like "Mr. Mime" and then the reporting DAO sanitizes the rest, but that adds a lot of complexity).execute(insert(), values)
syntax rather than the current.execute(insert().values(values)
syntax. AFAICT, this also necessitates storing the rows asdict
s rather than tuples. This is, honestly, a better syntax, as it'll be clearer from the tests what's being tested than how it is now where I'm randomly selecting by index. But it's also going to be a PITA to rewrite all the tests.