search

Baoqi / metabase-dremio-driver

Dremio driver for Metabase BI
Eclipse Public License 2.0
50 stars 10 forks source link

with dremio 21.1.1, schema filtering based on 'schema' connection parameter does not seem to work, and all dremio's datasets are synchronized #6

Closed fabrice-etanchaud closed 2 years ago

fabrice-etanchaud commented 2 years ago

Hi @Baoqi , I hope you are fine ? I just upgraded dremio to the last community version : 21.1.1. Your great driver does not seem to filter datasets anymore based on the 'schema' connection parameter. All datasets are synchronized, wathever the parameter's value, leading to huge load and performance loss.

I read the driver's code, but did not succeed in guessing where the problem is.

Thank you @Baoqi for your work on this driver !

Baoqi commented 2 years ago

Hi, @fabrice-etanchaud

I just made a commit for this issue, and make a new release 1.1.0 for this. (note: this is for Metabase 0.43.0+, you can check the README.md file)

metabase v0.43.0 add support for filtered-syncable-schemas

It turns out this is not related with Dremio, but with the Metabase.

In Metadata latest version (v0.43.0), it makes some changes, in commit https://github.com/metabase/metabase/commit/b6d542f82b6d504bb0540edc691e46e42efc9e13

which makes our previous "sync.i/syncable-schemas" not working. But metabase provide a better way for schema filters (it can select the schemas, plus specify which schemas to exclude) in that version. So, I remove the "sync.i/syncable-schemas" (which rely on current_schema, is not a good way, only a workaround). and use the "schema-filters" type field in metabase-plugin.yaml file.

Now, the UI becomes (the text is mainly because my browser's language settings, it should be English for English machines)

metabase1

As metabase's recent version changes the build to from "lein" to "clojure CLI tools", so the change and way to build will be different.

Baoqi commented 2 years ago

By the way, if you are using the latest Metabase 0.43.0, and you have previous metabase-dremio-driver at hand, and compiled it before. The only change to that (previous) version source code could be: the change in metabase-plugin.yaml as in this new commit, and then build it. (So that's built by "lein", instead of "clojure CLI tools")

fabrice-etanchaud commented 2 years ago

Hi @Baoqi , you are incredible ! Thank you !