dinedal / textql

Execute SQL against structured text like CSV or TSV
MIT License
9.05k stars 300 forks source link

too many SQL variables #111

Open razpeitia opened 5 years ago

razpeitia commented 5 years ago

I'm getting this error because I'm trying to insert around 2000 columns. Which is within the default limit for sqlite3.

I noticed that this is because the maximum number of SQL variables (999 by default).

and this is because the way you insert the rows. I think this should be done in batches. https://github.com/dinedal/textql/blob/master/storage/sqlite.go#L151-L160