Add more checks for spark-connect linter

databrickslabs / ucx

Your best companion for upgrading to Unity Catalog. UCX will guide you, the Databricks customer, through the process of upgrading your account, groups, workspaces, jobs etc. to Unity Catalog.

Other

196 stars 70 forks source link

Add more checks for spark-connect linter #2092

Closed vsevolodstep-db closed 1 week ago

vsevolodstep-db commented 1 week ago

Changes

Add more checks to detect code incompatibilities with UC Shared Clusters:

use of Python UDF unsupported eval types
spark.catalog.X APIs on DBR < 14.3
use of commandContext

Tests

[ ] manually tested
[x] added unit tests
[ ] added integration tests
[ ] verified on staging environment (screenshot attached)