search

databrickslabs / lsql

Lightweight SQL execution wrapper only on top of Databricks SDK
https://pypi.org/project/databricks-labs-lsql/
Other
12 stars 4 forks source link

Added catalog and schema parameters to execute and fetch. #90

Closed FastLee closed 6 months ago

FastLee commented 6 months ago

Added catalog and schema parameters to the fetch and execute command. Created unit and integration tests.

github-actions[bot] commented 6 months ago

❌ 19/20 passed, 1 failed, 2 skipped, 15m19s total

❌ test_overwrite: databricks.sdk.errors.platform.BadRequest: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: (671ms) ``` databricks.sdk.errors.platform.BadRequest: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have permission CREATE,USAGE on database `TEST_SCHEMA`. 17:18 DEBUG [databricks.sdk] Loaded from environment 17:18 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred 17:18 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred 17:18 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service 17:18 INFO [databricks.sdk] Using Databricks Metadata Service authentication [gw6] linux -- Python 3.10.14 /home/runner/work/lsql/lsql/.venv/bin/python 17:18 DEBUG [databricks.sdk] Loaded from environment 17:18 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred 17:18 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred 17:18 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service 17:18 INFO [databricks.sdk] Using Databricks Metadata Service authentication 17:18 DEBUG [databricks.labs.lsql.backends] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.TEST_SCHEMA.foo (first STRING NOT NULL, second BOOLEAN NOT... (18 more bytes) 17:18 DEBUG [databricks.labs.lsql.core] Executing SQL statement: CREATE TABLE IF NOT EXISTS hive_metastore.TEST_SCHEMA.foo (first STRING NOT NULL, second BOOLEAN NOT NULL) USING DELTA 17:18 DEBUG [databricks.sdk] POST /api/2.0/sql/statements/ > { > "format": "JSON_ARRAY", > "statement": "CREATE TABLE IF NOT EXISTS hive_metastore.TEST_SCHEMA.foo (first STRING NOT NULL, second BOOLEAN NOT... (18 more bytes)", > "warehouse_id": "TEST_DEFAULT_WAREHOUSE_ID" > } < 200 OK < { < "statement_id": "01eef827-92ef-141a-acea-cb4e27413b70", < "status": { < "error": { < "error_code": "BAD_REQUEST", < "message": "[INSUFFICIENT_PERMISSIONS] Insufficient privileges:\nUser does not have permission CREATE,USAGE o... (21 more bytes)" < }, < "state": "FAILED" < } < } 17:18 DEBUG [databricks.sdk] Loaded from environment 17:18 DEBUG [databricks.sdk] Ignoring pat auth, because metadata-service is preferred 17:18 DEBUG [databricks.sdk] Ignoring basic auth, because metadata-service is preferred 17:18 DEBUG [databricks.sdk] Attempting to configure auth: metadata-service 17:18 INFO [databricks.sdk] Using Databricks Metadata Service authentication 17:18 DEBUG [databricks.labs.lsql.backends] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.TEST_SCHEMA.foo (first STRING NOT NULL, second BOOLEAN NOT... (18 more bytes) 17:18 DEBUG [databricks.labs.lsql.core] Executing SQL statement: CREATE TABLE IF NOT EXISTS hive_metastore.TEST_SCHEMA.foo (first STRING NOT NULL, second BOOLEAN NOT NULL) USING DELTA 17:18 DEBUG [databricks.sdk] POST /api/2.0/sql/statements/ > { > "format": "JSON_ARRAY", > "statement": "CREATE TABLE IF NOT EXISTS hive_metastore.TEST_SCHEMA.foo (first STRING NOT NULL, second BOOLEAN NOT... (18 more bytes)", > "warehouse_id": "TEST_DEFAULT_WAREHOUSE_ID" > } < 200 OK < { < "statement_id": "01eef827-92ef-141a-acea-cb4e27413b70", < "status": { < "error": { < "error_code": "BAD_REQUEST", < "message": "[INSUFFICIENT_PERMISSIONS] Insufficient privileges:\nUser does not have permission CREATE,USAGE o... (21 more bytes)" < }, < "state": "FAILED" < } < } [gw6] linux -- Python 3.10.14 /home/runner/work/lsql/lsql/.venv/bin/python ```

Running from acceptance #62