allegro / turnilo

Business intelligence, data exploration and visualization web application for Druid, formerly known as Swiv and Pivot
https://allegro.github.io/turnilo/
Apache License 2.0
731 stars 175 forks source link

Adapting Turnilo to SQL backend? #674

Open EgorKraevTransferwise opened 4 years ago

EgorKraevTransferwise commented 4 years ago

Hi,

I am looking for a pivot table frontend to browse large datasets (~1B rows), and just love the usability of Turnilo (Superset's pivot table is a pain by comparison, and Redash's doesn't work at all for large datasets). However, I'd also like to use Clickhouse as a backend, along with Druid, and that's something that Turnilo doesn't support at the moment.

Are there any other open source pivot tables that are as nice to use as Turnilo and work with a SQL backend? And if there aren't, how feasible is it to let Turnilo support that? I'm not talking about any Plywood fanciness here, just basic display tables to choose from/groupby/filter/aggregate/display/update in realtime.

The two options I'm considering is a REST proxy that converts Druid requests (coming from Turnilo) to SQL and forwards them to Clickhouse, and converts responses to json - feels like it'd be quick to hack up but kind of ugly; and actually modifying Turnilo to support that usecase. In the latter case, would perhaps one of current contributors be interested in undertaking that work for pay (with results being open-sourced of course)?

Would really appreciate any advice.

adrianmroz commented 4 years ago

Hi,

We're not interested in supporting SQL backends in the foreseeable future. We want to focus our attention on Druid backend.

I'm not familiar with Clickhouse, but if I had to add SQL to turnilo I would fork it and around ClusterManager I would add ability to add sources that use plywood-postgress-requester and work from there. Turnilo heavily uses plywood at it's core and I'm afraid you couldn't go around it. And plywood -> druid translation isn't most efficient and probably you would add more bloat if you translate that druid queries to sql.

If you decide to make such fork, we would be happy to help with explanation of current code and some code reviews, but we can't promise that we would merge such fork to the upstream. We just can't use it internally and thus can't maintain it.

paoloxide commented 1 year ago

Hi @adrianmroz, I'd like to inquire if there has been plans now to include support to SQLs or ClickHouse?

adrianmroz-allegro commented 1 year ago

We're not interested in different backends than Druid. We just don't have bandwidth for that in our team.