search

markhoerth / dremio_client

Apache License 2.0
31 stars 26 forks source link

Feature request: configure Dremio cluster #154

Closed louis030195 closed 3 years ago

louis030195 commented 3 years ago

Description

It's a feature request, having the possibility to configure a Dremio cluster using this client (users, datasets, datasources ..)

rymurr commented 3 years ago

Hey @louis030195 thanks for the interest!

What do you mean when you say configure? Is there any specific operation you had in mind?

The dremio_client can create and modify sources/spaces/folders/datasets but not users. There isn't a REST endpoint to create users in Dremio as it typically delegates that work to the backing oauth/ldap store.

Unfortunately the documentation is rather poor on how to do this so I will try and make it more clear.

louis030195 commented 3 years ago

Thanks for the quick answer ! I was referring to Dremio users and datasource creation and configuration (for example say I want a S3 datasource with X credentials ...) I couldn't find anything for that or am I wrong ? It would be quite useful for us to have a fully automated cluster deployment.

rymurr commented 3 years ago

Users aren't supported currently as there are no rest apis for it. Unfortunately.

To add a source you would do something like (forgive me, its been a while since I have used this and the syntax may not be 100%):

catalog = init()
catalog.data.add({source})
catalog.data.source.commit()

Where source would be (from https://docs.dremio.com/rest-api/catalog/container-source.html) and in the example above the name is source. Currently there is nothing in the tool to make constructing the source dict easier.