search

RootDBApp / rootdb

Main repository containing all releases.
https://www.rootdb.fr
GNU Affero General Public License v3.0
13 stars 0 forks source link

Extract (tableau hyper-like) feature #12

Open simonaubertbd opened 2 months ago

simonaubertbd commented 2 months ago

Hello,

From what I understand, we can now exploit purely databases. However, a lot of users have their data in files or in databases that are not optimized at all for OLAP (Thinking of a postgresl, Mysql, Mariadb, etc...).

The idea would be to : -extract the said data from this "slow" datasource -write it in a "fast" internal datasource. Duckdb would be really great for that kind of stuff. -allow to schedule the refresh and monitor the task refresh.

Best regards,

Simon

spo-ijaz commented 1 month ago

Hi @simonaubertbd

Sorry, I completely missed this last suggestion !

Thanks again for this suggestion, it seems interesting, since we can request Duckdb in SQL from what I see. CSV/Excel imports are already planned, with asynchronous import for big files. But I since we will include a support for DuckDb, because I can directly see where I can use that - meaning, I need to test that on with a real use case.