Evaluate replacing the system table connector with the DB SDK - Githubissues

databrickslabs / overwatch

Capture deep metrics on one or all assets within a Databricks workspace

Other

230 stars 65 forks source link

Evaluate replacing the system table connector with the DB SDK #1295

Open gueniai opened 2 months ago

gueniai commented 2 months ago

[x] Execute a query using the Statement Execution API in a local prototype
[x] Expose Spark's ArrowConverters.toDataFrame private method
[ ] Decode the Arrow payload retrieved from a statement execution external link
[ ] Implement iteration over external links
[ ] Move logic into Overwatch codebase