New Source: Dremio

[kurousa]

Tell us about the new connector you’d like to have

• Which source and which destination?

  • Source

• Do you need a specific version of the underlying data source e.g: you specifically need support for an older version of the API or DB?

  • Want to support for since 4.0.x.

Describe the context around this new connector

• Why do you need this integration? How does your team intend to use the data? This helps us understand the use case.

  • We use this to DWH and analyze the data by other plaform.

• How often do you want to run syncs?

  • It's not clear, but it needs to be synchronized at regular intervals.

• If this is an API source connector, which entities/endpoints do you need supported?

  • We want run query on the Dremio, and retrieve the result by airbyte.

• If the connector is for a paid service, can we name you as a mutual user when we subscribe for an account? Which company should we name?

  • We probably not possible.

Describe the alternative you are considering or using

What are you considering doing if you don’t have this integration through Airbyte?

• Currently no alternative.

  Are you willing to submit a PR?

  Authentication Option | Login Credentials - UI | Login Credentials - ODBC/JDBC/REST -- | -- | -- Azure AD | Single Sign-On & Personal Access Token | Personal Access Token LDAP | Username / Password &Personal Access Token | Username / Password &Personal Access Token Local | Username / Password & | Username / Password &Personal Access Token OpenID | Single Sign-On &Personal Access Token | Personal Access Token

  is there an SDK available? Can we connect with HTTP requests directly? SDK https://www.dremio.com/blog/building-a-javascript-sdk-for-dremio/

  If this is a database/datalake/pubsub source, does it have a JDBC driver or Java SDK? _yes https://docs.dremio.com/software/drivers/dremio-jdbc-driver/

  Creating an account self-service free account

  Will it require some work through the UI? If so, what is required? Yes. When connecting sources

  Has Sandbox environment? No How to populate the account with data UI: As datalake dremio supports a variety of dat sources: https://docs.dremio.com/software/data-sources/

  Available streams for sync Accounts API Catalog API Job API Node Collections API Privileges/Grant API (16.0+) Reflection API Role API (18.0+) Source API SQL API Token API User API Vote API Workload Management API

  Integration supports incremental sync? Incremental is a property at the PDS level, so you can set it there

  http://docs.dremio.com/rest-api/catalog/dataset/ 24

  To disable automatic refresh, check never refresh, never expire at the PDS the VDS is built on via the UI

  Other information/blockers Setting up account needs a lot of actions on AWS side https://docs.dremio.com/cloud/signing-up/prerequisites-and-permissions/

[lazebnyi]

Scoping Document for Dremio