search

AlexR2D2 / metabase_duckdb_driver

Metabase DuckDB Driver shipped as 3rd party plugin
Apache License 2.0
70 stars 21 forks source link

Request for information #2

Closed seajhawk closed 1 year ago

seajhawk commented 1 year ago

I would love to talk to you about your experiences with DuckDB and Metabase!

I'm considering switching from Postgres to DuckDB as my database is purely analytical in nature. I'm hoping to drop the hosting cost for my Postgres server, reduce space through columnar compression, and improving query performance would be great!

If you don't mind sharing a bit, please let me know!

AlexR2D2 commented 1 year ago

Hello, i have no yet experience with Metabase and DuckDB, but i think it will be interested to use DuckDB with Metabase in some usecases. For example, you can store data in .parquet format and Metabase with duckdb plugin is only you need to analyse this data. Not requiered any additional database servers or analytilical services. Just raw data files and Metabase app. But i think this thing has limits, it will be hard to use it with the huge amount of distributed on different hosts data.

As for the transition from Postgres to DuckDB, i think it make sense if you data fits into the one host, because DuckDB has no built-in auto-sharding or replication or if it is suitable for you to make separate queries to different servers with data.

seajhawk commented 1 year ago

I agree - most of my data is 10's of millions of rows or less, and I currently have different datasets in the same Postgres server, but they are generally unrelated and can stand alone with their own reports, etc. My next hurdle is to figure out how I want to make the duckdb file(s) available to my instances of Metabase which run in containers... something new to learn every day.

I'm also looking forward to testing the performance difference between Postgres & DuckDB. Based on playing with DuckDB locally, I'm anticipating it will speed things up quite a bit.