datacontract / datacontract-cli

CLI to manage your datacontract.yaml files
https://cli.datacontract.com
Other
353 stars 60 forks source link

sqlserver not supported by datacontract CLI #229

Closed mathTosetto closed 1 month ago

mathTosetto commented 1 month ago

When picking sqlserver as my server type I get the following error during the tests stage.

Example:

servers:
  production:
    type: sqlserver
    host: fake-sqlserver.example.com
    port: 1433
    database: database
    schema: dbo

Warning: Server type sqlserver not yet supported by datacontract CLI

Any idea to fix it?

jochenchrist commented 1 month ago

sqlserver support is not yet released and is planned for v0.10.5, will be released probably this week (https://github.com/datacontract/datacontract-cli/blob/main/CHANGELOG.md)

You could test it with the latest development version: https://github.com/datacontract/datacontract-cli?tab=readme-ov-file#development-setup

mathTosetto commented 1 month ago

sqlserver support is not yet released and is planned for v0.10.5, will be released probably this week (https://github.com/datacontract/datacontract-cli/blob/main/CHANGELOG.md)

You could test it with the latest development version: https://github.com/datacontract/datacontract-cli?tab=readme-ov-file#development-setup

Great! Thanks for the quick reply.

By the way, I just came across this library today and it's a lifesaver. Thanks for that 😁

jochenchrist commented 1 month ago

data contract v0.10.6 has been released 🎉.

@Thrit can you test if it works for you?

mathTosetto commented 1 month ago

data contract v0.10.6 has been released 🎉.

@Thrit can you test if it works for you?

Happy days 🎉

Sure, I’ll get back to you soon.

mathTosetto commented 1 month ago

So, I'm still facing some issues when running the test command.

This is my environment:

My datacontract has these values for the servers section:

servers:
  production:
    type: sqlserver
    host: sqlserver-fake-host.database.windows.net
    port: 1433
    database: database
    schema: dbo

Running the command datacontract test datacontract.yaml I get the following message: ERROR:soda.scan:[21:16:16] Could not connect to data source "sqlserver": Encountered a problem while trying to connect to sqlserver: can only concatenate str (not "NoneType") to str.

I was debugging the functions that led to the error and encountered the moment when the variable returned None.

The error occurs in the _parse_configuration_yaml_str function (Line 180) in the scan.py.

When line 184 runs, it returns:

data_source sqlserver:
  database: database
  driver: null
  encrypt: true
  host: sqlserver-fake-host.database.windows.net
  password: ''
  port: '1433'
  schema: dbo
  trust_server_certificate: false
  trusted_connection: false
  type: sqlserver
  username: ''

After instancing the ConfigurationParser, at line 191 is where I get the NoneType value.

I will come back to the code afterwards.

jochenchrist commented 1 month ago

Do you need to specific the driver?

servers:
  production:
    type: sqlserver
    host: sqlserver-fake-host.database.windows.net
    port: 1433
    database: database
    schema: dbo
    driver: ODBC Driver 18 for SQL Server
mathTosetto commented 1 month ago

Do you need to specific the driver?

servers:
  production:
    type: sqlserver
    host: sqlserver-fake-host.database.windows.net
    port: 1433
    database: database
    schema: dbo
    driver: ODBC Driver 18 for SQL Server

I made some progress 🎉 I didn't realize that the soda really looks up to a valid db. I thought adding any value could be accepted.

Therefore, I have created a simple sql server db on a Dockerfile and then added the Driver value as you suggested. Now, I'm getting this error which is related to the TrustServerCertificate

ERROR:soda.scan:[19:41:26] Could not connect to data source "sqlserver": Encountered a problem while trying to connect to sqlserver: ('08001', '[08001] [Microsoft][ODBC Driver 18 for SQL Server]SSL Provider: [error:0A000086:SSL routines::certificate verify failed:self-signed certificate] (-1) (SQLDriverConnect)')

I'm running a few more tests to manage this error.

jochenchrist commented 1 month ago

Have a look at the env variables listed at: https://cli.datacontract.com/#sqlserver that might be relevant for you:

DATACONTRACT_SQLSERVER_USERNAME 
DATACONTRACT_SQLSERVER_PASSWORD 
DATACONTRACT_SQLSERVER_TRUSTED_CONNECTION
DATACONTRACT_SQLSERVER_TRUST_SERVER_CERTIFICATE
DATACONTRACT_SQLSERVER_ENCRYPTED_CONNECTION
mathTosetto commented 1 month ago

Done!

image

The only thing I'd say is to add the Driver value and its description in the documentation when you have a chance.

https://datacontract.com/#quality-object

image

Thanks for all the support 😄

jochenchrist commented 1 month ago

image

Thanks for testing :)