pshapiro / rankenberry

Simple rank tracking off of 3rd party api with advanced data features
GNU Affero General Public License v3.0
38 stars 10 forks source link

Investigate DuckDB #11

Open derekperkins opened 4 days ago

derekperkins commented 4 days ago

SQLite will choke pretty quickly on analytical queries. DuckDB is essentially SQLite as an analytical, columnar database. Putting reporting there will help SQLite stay viable longer for interactions. If you wanted to get extra fancy, there's an official WASM port that runs client side in the browser, that should be sufficient for most of the data sizes that SQLite would support.

derekperkins commented 4 days ago

To summarize our conversation for this issue, I would still use SQLite for all the configuration data, and only put the actual ranking data into DuckDB.