sodadata / soda-core

:zap: Data quality testing for the modern data stack (SQL, Spark, and Pandas) https://www.soda.io
https://go.soda.io/core-docs
Apache License 2.0
1.9k stars 208 forks source link

How to connect to Dremio Cloud? #2007

Closed JorisTruong closed 9 months ago

JorisTruong commented 9 months ago

Hi,

I am trying to try out soda-core with Dremio Cloud. According to Soda's documentation, the connection configuration should look like this:

data_source my_datasource_name:
  type: dremio
  host: 127.0.0.1
  port: 5432
  username: simple
  password: simple_pass
  schema: public

The Dremio's documentation suggests to use data.dremio.cloud as the host and a token instead of username/password.

Any advice? I also have installed the ODBC driver for Arrow Flight SQL as it looks like this is what is used the the source code, but it has not been working for me so far.

Here is a code snippet with the error log:

scan = Scan()
scan.set_data_source_name("cost_analysis")
scan.add_configuration_yaml_str(
    """
    data_source cost_analysis:
      type: dremio
      host: data.dremio.cloud
      port: 443
      token: <personal access token>
"""
)
scan.add_sodacl_yaml_str(sodacl_yaml_str)
scan.execute()
scan.assert_no_checks_fail()

Encountered a problem while trying to connect to dremio: can only concatenate str (not "int") to str

tools-soda commented 9 months ago

SAS-2800

m1n0 commented 9 months ago

Hi! This is currently not supported, but it should be a fairly straightforward change. Would you be open to help and contribute token support?

JorisTruong commented 9 months ago

Closed as #2009 has been merged