Error with clickhouse-metabase-driver 1.50 and metabase 0.50.1

tuandn8 commented 3 weeks ago

Describe the bug

No method in multimethod 'date' for dispatch value: [:clickhouse :day]

Steps to reproduce

Use new Metabase driver v1.50.0
Upgrade from metabase v0.49.14 to v0.50.1

Expected behaviour

Error log

When start Metabase

2024-06-13 02:29:02,310 ERROR driver.util :: Failed to connect to Database
java.lang.IllegalArgumentException: No method in multimethod 'connection-details->spec' for dispatch value: :clickhouse
    at clojure.lang.MultiFn.getFn(MultiFn.java:156)
    at clojure.lang.MultiFn.invoke(MultiFn.java:233)
    at metabase.driver.sql_jdbc.connection$do_with_connection_spec_for_testing_connection$fn__80912.invoke(connection.clj:313)
    at metabase.util.ssh$do_with_ssh_tunnel.invokeStatic(ssh.clj:165)
    at metabase.util.ssh$do_with_ssh_tunnel.invoke(ssh.clj:154)
    at metabase.driver.sql_jdbc.connection$do_with_connection_spec_for_testing_connection.invokeStatic(connection.clj:312)
    at metabase.driver.sql_jdbc.connection$do_with_connection_spec_for_testing_connection.invoke(connection.clj:308)
    at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invokeStatic(connection.clj:337)
    at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invoke(connection.clj:333)
    at metabase.driver.sql_jdbc$fn__108704.invokeStatic(sql_jdbc.clj:49)
    at metabase.driver.sql_jdbc$fn__108704.invoke(sql_jdbc.clj:47)
    at clojure.lang.MultiFn.invoke(MultiFn.java:234)
    at metabase.driver.util$can_connect_with_details_QMARK_$fn__56069.invoke(util.clj:149)
    at clojure.core$binding_conveyor_fn$fn__5823.invoke(core.clj:2047)
    at clojure.lang.AFn.call(AFn.java:18)
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)
2024-06-13 02:29:02,312 WARN task.sync-databases :: Cannot sync Database Clickhouse view_portal: No method in multimethod 'connection-details->spec' for dispatch value: :clickhouse
clojure.lang.ExceptionInfo: No method in multimethod 'connection-details->spec' for dispatch value: :clickhouse {:message "No method in multimethod 'connection-details->spec' for dispatch value: :clickhouse"}
    at metabase.driver.util$can_connect_with_details_QMARK_.invokeStatic(util.clj:166)
    at metabase.driver.util$can_connect_with_details_QMARK_.doInvoke(util.clj:137)
    at clojure.lang.RestFn.invoke(RestFn.java:442)
    at metabase.task.sync_databases$sync_and_analyze_database_STAR__BANG_$fn__108487$fn__108488.invoke(sync_databases.clj:82)
    at metabase.task.sync_databases$sync_and_analyze_database_STAR__BANG_$fn__108487.invoke(sync_databases.clj:81)
    at metabase.task.sync_databases$sync_and_analyze_database_STAR__BANG_.invokeStatic(sync_databases.clj:78)
    at metabase.task.sync_databases$sync_and_analyze_database_STAR__BANG_.invoke(sync_databases.clj:71)
    at metabase.task.sync_databases$sync_and_analyze_database_BANG_.invokeStatic(sync_databases.clj:107)
    at metabase.task.sync_databases$sync_and_analyze_database_BANG_.invoke(sync_databases.clj:95)
    at metabase.task.sync_databases.SyncAndAnalyzeDatabase.execute(sync_databases.clj:112)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
Caused by: java.lang.IllegalArgumentException: No method in multimethod 'connection-details->spec' for dispatch value: :clickhouse
    at clojure.lang.MultiFn.getFn(MultiFn.java:156)
    at clojure.lang.MultiFn.invoke(MultiFn.java:233)
    at metabase.driver.sql_jdbc.connection$do_with_connection_spec_for_testing_connection$fn__80912.invoke(connection.clj:313)
    at metabase.util.ssh$do_with_ssh_tunnel.invokeStatic(ssh.clj:165)
    at metabase.util.ssh$do_with_ssh_tunnel.invoke(ssh.clj:154)
    at metabase.driver.sql_jdbc.connection$do_with_connection_spec_for_testing_connection.invokeStatic(connection.clj:312)
    at metabase.driver.sql_jdbc.connection$do_with_connection_spec_for_testing_connection.invoke(connection.clj:308)
    at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invokeStatic(connection.clj:337)
    at metabase.driver.sql_jdbc.connection$can_connect_QMARK_.invoke(connection.clj:333)
    at metabase.driver.sql_jdbc$fn__108704.invokeStatic(sql_jdbc.clj:49)
    at metabase.driver.sql_jdbc$fn__108704.invoke(sql_jdbc.clj:47)
    at clojure.lang.MultiFn.invoke(MultiFn.java:234)
    at metabase.driver.util$can_connect_with_details_QMARK_$fn__56069.invoke(util.clj:149)
    at clojure.core$binding_conveyor_fn$fn__5823.invoke(core.clj:2047)
    at clojure.lang.AFn.call(AFn.java:18)
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)

When execute a query

2024-06-13 02:31:42,037 ERROR middleware.catch-exceptions :: Error processing query: No method in multimethod 'date' for dispatch value: [:clickhouse :day]
{:database_id 3,
 :started_at #t "2024-06-13T02:31:41.946591Z[GMT]",
 :via
 [{:status :failed,
   :class clojure.lang.ExceptionInfo,
   :error "Error compiling query: No method in multimethod 'date' for dispatch value: [:clickhouse :day]",
   :stacktrace
   ["--> query_processor.compile$compile_preprocessed$fn__65860.invoke(compile.clj:25)"
    "query_processor.setup$do_with_qp_setup.invokeStatic(setup.clj:225)"
    "query_processor.setup$do_with_qp_setup.invoke(setup.clj:216)"
    "query_processor.compile$compile_preprocessed.invokeStatic(compile.clj:21)"
    "query_processor.compile$compile_preprocessed.invoke(compile.clj:17)"
    "query_processor.compile$attach_compiled_query.invokeStatic(compile.clj:43)"
    "query_processor.compile$attach_compiled_query.invoke(compile.clj:35)"
    "query_processor$process_query_STAR__STAR_.invokeStatic(query_processor.clj:45)"
    "query_processor$process_query_STAR__STAR_.invoke(query_processor.clj:43)"
    "query_processor.middleware.enterprise$fn__64281$handle_audit_app_internal_queries__64282$fn__64284.invoke(enterprise.clj:96)"
    "query_processor.middleware.enterprise$handle_audit_app_internal_queries_middleware$fn__64292.invoke(enterprise.clj:103)"
    "query_processor.middleware.process_userland_query$process_userland_query_middleware$fn__75392.invoke(process_userland_query.clj:182)"
    "query_processor.middleware.catch_exceptions$catch_exceptions$fn__75461.invoke(catch_exceptions.clj:128)"
    "query_processor$process_query$fn__75498.invoke(query_processor.clj:78)"
    "query_processor.setup$do_with_canceled_chan$fn__64696.invoke(setup.clj:189)"
    "query_processor.setup$do_with_database_local_settings$fn__64691.invoke(setup.clj:181)"
    "query_processor.setup$do_with_driver$fn__64686$fn__64687.invoke(setup.clj:166)"
    "driver$do_with_driver.invokeStatic(driver.clj:104)"
    "driver$do_with_driver.invoke(driver.clj:99)"
    "query_processor.setup$do_with_driver$fn__64686.invoke(setup.clj:165)"
    "query_processor.setup$do_with_metadata_provider$fn__64679$fn__64682.invoke(setup.clj:151)"
    "query_processor.store$do_with_metadata_provider.invokeStatic(store.clj:171)"
    "query_processor.store$do_with_metadata_provider.invoke(store.clj:151)"
    "query_processor.store$do_with_metadata_provider.invokeStatic(store.clj:160)"
    "query_processor.store$do_with_metadata_provider.invoke(store.clj:151)"
    "query_processor.setup$do_with_metadata_provider$fn__64679.invoke(setup.clj:150)"
    "query_processor.setup$do_with_resolved_database$fn__64673.invoke(setup.clj:128)"
    "query_processor.setup$do_with_qp_setup.invokeStatic(setup.clj:232)"
    "query_processor.setup$do_with_qp_setup.invoke(setup.clj:216)"
    "query_processor$process_query.invokeStatic(query_processor.clj:76)"
    "query_processor$process_query.invoke(query_processor.clj:69)"
    "query_processor.card$process_query_for_card_default_qp.invokeStatic(card.clj:170)"
    "query_processor.card$process_query_for_card_default_qp.invoke(card.clj:166)"
    "query_processor.card$process_query_for_card_default_run_fn$fn__81457$fn__81458.invoke(card.clj:177)"
    "query_processor.streaming$_streaming_response$fn__68005$fn__68006$fn__68007.invoke(streaming.clj:175)"
    "query_processor.streaming$_streaming_response$fn__68005$fn__68006.invoke(streaming.clj:174)"
    "query_processor.streaming$do_with_streaming_rff.invokeStatic(streaming.clj:165)"
    "query_processor.streaming$do_with_streaming_rff.invoke(streaming.clj:152)"
    "query_processor.streaming$_streaming_response$fn__68005.invoke(streaming.clj:171)"
    "async.streaming_response$do_f_STAR_.invokeStatic(streaming_response.clj:68)"
    "async.streaming_response$do_f_STAR_.invoke(streaming_response.clj:66)"
    "async.streaming_response$do_f_async$task__52126.invoke(streaming_response.clj:87)"],
   :error_type :driver,
   :ex-data

Configuration

Environment

metabase-clickhouse-driver version: v1.50.0
metabase-clickhouse-driver configuration: default setting
Metabase version: v0.50.1
OS: docker

ClickHouse server

ClickHouse Server version: v22.18

phantom943 commented 3 weeks ago

Second it. Same happens on higher metabase versions (we are running Metabase 0.50.3 and adapter does not seem to be working correctly there)

slvrtrn commented 3 weeks ago

This is odd cause the code is there: https://github.com/ClickHouse/metabase-clickhouse-driver/blob/master/src/metabase/driver/clickhouse_qp.clj#L101-L103

Another weird thing in the stack trace:

java.lang.IllegalArgumentException: No method in multimethod 'connection-details->spec' for dispatch value: :clickhouse

And it is there as well: https://github.com/ClickHouse/metabase-clickhouse-driver/blob/master/src/metabase/driver/clickhouse.clj#L94-L95

I tried it with a clean instance, and it works as expected (can create a connection, scan tables, execute queries, etc).

I saw similar errors in the stack trace when an outdated jar was used with an incompatible MB version, though.

So, regarding this:

Upgrade from metabase v0.49.14 to v0.50.1

Does it happen only if you already have a CH connection with an older version of MB and then upgrade it (both MB and the driver)?

phantom943 commented 3 weeks ago

I think it might be a caching problem, at least for me it was. I checked - and my adapter version was still on 1.5 or something like this. I forced updating the adapter - and now the same dashboard with a date filter works as expected, without that bug.

slvrtrn commented 3 weeks ago

I checked - and my adapter version was still on 1.5 or something like this. I forced updating the adapter - and now the same dashboard with a date filter works as expected, without that bug.

Can you please share the steps that need to be taken in this scenario? I can then add them to the README.

phantom943 commented 3 weeks ago

I checked - and my adapter version was still on 1.5 or something like this. I forced updating the adapter - and now the same dashboard with a date filter works as expected, without that bug.

Can you please share the steps that need to be taken in this scenario? I can then add them to the README.

Well, I think that highly depends on your deployment strategy of Metabase. We are deploying via CI/CD to Kubernetes, and we just had to override the use_cache variable in our CI step of downloading the Clickhouse Metabase driver version during the build step.

slvrtrn commented 3 weeks ago

@phantom943, thanks!

@tuandn8, can you please check if it's a similar cause in your case? Maybe an outdated jar was somehow mounted to your Docker container or the Docker build was using a cached layer with the old version of the driver instead of a new one.

moose-code commented 3 weeks ago

Hi all.

Just want to chime in and say we are seeing some weird behaviour where the total number of entities in our table seems to fluctuate between 20k then jumps to 80k, and so on so forth.

I'm not sure if this is also related?

slvrtrn commented 2 weeks ago

Just want to chime in and say we are seeing some weird behaviour where the total number of entities in our table seems to fluctuate between 20k then jumps to 80k, and so on so forth.

Not related to this one.

However, if you are using a cluster deployment, you could check the tables on individual nodes, as it could be that the data is not replicated (yet).

If the data is properly replicated, or it happens with a single-node deployment, then it is a separate issue.

tuandn8 commented 2 weeks ago

I run a single instance of Metabase and use Docker mount plugins from host to container.

I migrate from Metabase v49.0 to v50.0 and still use old driver v1.4.0. It returns error when run any Question with Clickhouse datasource. Then I read your document and move Clickhouse driver to v1.5.1, But it still happened.

slvrtrn commented 2 weeks ago

I migrate from Metabase v49.0 to v50.0 and still use old driver v1.4.0. It returns error when run any Question with Clickhouse datasource. Then I read your document and move Clickhouse driver to v1.5.1, But it still happened.

MB 0.50.x should be used with the driver version 1.50.0.

See https://github.com/ClickHouse/metabase-clickhouse-driver?tab=readme-ov-file#choosing-the-right-version

slvrtrn commented 4 days ago

I am closing this for now; please feel free to reopen or create a new one if the issue persists.

ClickHouse / metabase-clickhouse-driver