search

orenfromberg / lxxproject

generating a sqlite database with entries in the original greek from freely available resources on the web.
4 stars 3 forks source link

Insert in Chunks #3

Closed jcuenod closed 8 years ago

jcuenod commented 8 years ago

This also adds the root word (but it's technically flawed because it includes only the last column in compound words - I'm not sure how I want to fix this but I'm thinking the best bet is to redo the regex and merge the morphology columns for when I do that).

orenfromberg commented 8 years ago

I'm guessing this PR is for these issues https://github.com/orenfromberg/lxxproject/issues/1 https://github.com/orenfromberg/lxxproject/issues/2 I'll close the issues.

looks good, thanks for the contribution! and it runs FAST! love it.

jcuenod commented 8 years ago

Magic number 499 (which is 500 when chunkCounter starts at 0) is because Sqlite has a limit of 500 insertions that it can do at a time.