datacontract / datacontract-cli

CLI to manage your datacontract.yaml files
https://cli.datacontract.com
Other
351 stars 60 forks source link

Reg: Trino integration #264

Open bazooka720 opened 1 week ago

bazooka720 commented 1 week ago

We plan to use it as a library vs CLI and wondering if you offer support for Trino? Something thet will connect to Trino REST API? We have some views and tables populated by those queries (we have RBAC established and so all connections to trio needs specific user params)

stefannegele commented 1 week ago

Hi @bazooka720, thanks for the issue. Can you please add some more information on the specific user params you need?

bazooka720 commented 1 week ago

subdomain/url/cluster username of user making query password of user making query query string catalog and schema might be sent as independant variables, if not trino will extract them from the query (optional)

bazooka720 commented 1 week ago

Hi. Curious any update on this?

jochenchrist commented 1 week ago

Let's discuss the required new server properties for the Data Contract Specification.

Proposal:

servers:
  production:
    type: trino
    host: 127.0.0.1
    port: 5432
    catalog: hive
    schema: my_schema

For connecting with the CLI, the credentials would be ENV variables, like these:

DATACONTRACT_TRINO_USERNAME=username
DATACONTRACT_TRINO_PASSWORD=password

The query will be composed in datacontract cli that executes the tests.

Any important things missing here?

bazooka720 commented 1 week ago

What if we do via the library?

On Wed, Jun 19, 2024, 12:00 AM jochenchrist @.***> wrote:

Let's discuss the required new server properties for the Data Contract Specification.

Proposal:

servers: production: type: trino host: 127.0.0.1 port: 5432 catalog: hive schema: my_schema

For connecting with the CLI, the credentials would be ENV variables, like these:

DATACONTRACT_TRINO_USERNAME=username DATACONTRACT_TRINO_PASSWORD=password

The query will be composed in datacontract cli that executes the tests.

Any important things missing here?

— Reply to this email directly, view it on GitHub https://github.com/datacontract/datacontract-cli/issues/264#issuecomment-2177576883, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGVA56UKMJNZZZNSVCGPD63ZID65VAVCNFSM6AAAAABJKVUYOWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZXGU3TMOBYGM . You are receiving this because you were mentioned.Message ID: @.***>

jochenchrist commented 1 week ago

This would also apply for the Python library. You have the server details in the contract and read credentials from ENV variables.

bazooka720 commented 6 days ago

Sounds good

stefannegele commented 5 days ago

Quick update:

I drafted PRs for the spec and for the CLI to support trino. Unfortunatly, I still have an issue in the unit test and don't have time to address it until next week.

https://github.com/datacontract/datacontract-specification/pull/73 https://github.com/datacontract/datacontract-cli/pull/278

bazooka720 commented 1 day ago

Is it planned to be merged soon?

stefannegele commented 1 day ago

I expect to finish it this week