Added escape_name function to escape individual SQL names and escape_full_name function to escape dot-separated full names (#316). Two new functions, escape_name and escape_full_name, have been added to the databricks.labs.lsql.escapes module for escaping SQL names. The escape_name function takes a single name as an input and returns it enclosed in backticks, while escape_full_name handles dot-separated full names by escaping each individual component. These functions have been ported from the databrickslabs/ucx repository and are designed to provide a consistent way to escape names and full names in SQL statements, improving the robustness of the system by preventing issues caused by unescaped special characters in SQL names. The test suite includes various cases, including single names, full names with different combinations of escaped and unescaped components, and special characters, with a specific focus on the scenario where the column name contains a period.
Bump actions/checkout from 4.2.0 to 4.2.1 (#304). In this pull request, the actions/checkout dependency is updated from version 4.2.0 to 4.2.1 in the .github/workflows/release.yml file. This update includes a new feature where refs/* are checked out by commit if provided, falling back to the ref specified by the @orhantoy user. This change improves the flexibility of the action, allowing users to specify a commit or branch for checkout. The pull request also introduces a new contributor, @Jcambass, who added a workflow file for publishing releases to an immutable action package. The commits for this release include changes to prepare for the 4.2.1 release, add a workflow file for publishing releases, and check out other refs/* by commit if provided, falling back to ref. This pull request has been reviewed and approved by Dependabot.
Bump actions/checkout from 4.2.1 to 4.2.2 (#310). This is a pull request to update the actions/checkout dependency from version 4.2.1 to 4.2.2, which includes improvements to the url-helper.ts file that now utilize well-known environment variables and expanded unit test coverage for the isGhes function. The actions/checkout action is commonly used in GitHub Actions workflows for checking out a repository at a specific commit or branch. The changes in this update are internal to the actions/checkout action and should not affect the functionality of the project utilizing this action. The pull request also includes details on the commits and compatibility score for the upgrade, and reviewers can manage and merge the request using Dependabot commands once the changes have been verified.
Bump databrickslabs/sandbox from acceptance/v0.3.0 to 0.3.1 (#307). In this release, the databrickslabs/sandbox dependency has been updated from version acceptance/v0.3.0 to 0.3.1. This update includes previously tagged commits, bug fixes for git-related libraries, and resolution of the unsupported protocol scheme error. The README has been updated with more information on using the databricks labs sandbox command, and installation instructions have been improved. Additionally, there have been dependency updates for go-git libraries and golang.org/x/crypto in the /go-libs and /runtime-packages directories. New commits in this release allow larger logs from acceptance tests and implement experimental OIDC refresh functionality. Ignore conditions have been applied to prevent conflicts with previous versions of the dependency. This update is recommended for users who want to take advantage of the latest bug fixes and improvements.
Bump databrickslabs/sandbox from acceptance/v0.3.1 to 0.4.2 (#315). In this release, the databrickslabs/sandbox dependency has been updated from version acceptance/v0.3.1 to 0.4.2. This update includes bug fixes, dependency updates, and additional go-git libraries. Specifically, the Run integration tests job in the GitHub Actions workflow has been updated to use the new version of the databrickslabs/sandbox/acceptance Docker image. The updated version also includes install instructions, usage instructions in the README, and a modification to provide more git-related libraries. Additionally, there were several updates to dependencies, including golang.org/x/crypto version 0.16.0 to 0.17.0. Dependabot, a tool that manages dependencies in GitHub projects, is responsible for the update and provides instructions for resolving any conflicts or merging the changes into the project. This update is intended to improve the functionality and reliability of the databrickslabs/sandbox dependency.
Deprecate Row.as_dict() (#309). In this release, we are introducing a deprecation warning for the as_dict() method in the Row class, which will be removed in favor of the asDict() method. This change aims to maintain consistency with Spark's Row behavior and prevent subtle bugs when switching between different backends. The deprecation warning will be implemented using Python's warnings mechanism, including the new annotation in Python 3.13 for static code analysis. The existing functionality of fetching values from the database through StatementExecutionExt remains unchanged. We recommend that clients update their code to use .asDict() instead of .as_dict() to avoid any disruptions. A new test case test_row_as_dict_deprecated() has been added to verify the deprecation warning for Row.as_dict().
Minor improvements for .save_table(mode="overwrite") (#298). In this release, the .save_table() method has been improved, particularly when using the overwrite mode. If no rows are supplied, the table will now be truncated, ensuring consistency with the mock backend behavior. This change has been optimized for SQL-based backends, which now perform truncation as part of the insert for the first batch. Type hints on the abstract method have been updated to match the concrete implementations. Unit tests and integration tests have been updated to cover the new functionality, and new methods have been added to test the truncation behavior in overwrite mode. These improvements enhance the consistency and efficiency of the .save_table() method when using overwrite mode across different backends.
Updated databrickslabs/sandbox requirement to acceptance/v0.3.0 (#305). In this release, we have updated the requirement for the databrickslabs/sandbox package to version acceptance/v0.3.0 in the downstreams.yml file. This update is necessary to use the latest version of the package, which includes several bug fixes and dependency updates. The databrickslabs/sandbox package is used in the acceptance tests, which are run as part of the CI/CD pipeline. It provides a set of tools and utilities for developing and testing code in a sandbox environment. The changelog for this version includes the addition of install instructions, more git-related libraries, and the modification of the README to include information about how to use it with the databricks labs sandbox command. Specifically, the version of the databrickslabs/sandbox package used in the acceptance job has been updated from acceptance/v0.1.4 to acceptance/v0.3.0, allowing the integration tests to be run using the latest version of the package. The ignore conditions for this PR ensure that Dependabot will resolve any conflicts that may arise and can be manually triggered with the @dependabot rebase command.
escape_name
function to escape individual SQL names andescape_full_name
function to escape dot-separated full names (#316). Two new functions,escape_name
andescape_full_name
, have been added to thedatabricks.labs.lsql.escapes
module for escaping SQL names. Theescape_name
function takes a single name as an input and returns it enclosed in backticks, whileescape_full_name
handles dot-separated full names by escaping each individual component. These functions have been ported from thedatabrickslabs/ucx
repository and are designed to provide a consistent way to escape names and full names in SQL statements, improving the robustness of the system by preventing issues caused by unescaped special characters in SQL names. The test suite includes various cases, including single names, full names with different combinations of escaped and unescaped components, and special characters, with a specific focus on the scenario where the column name contains a period.actions/checkout
dependency is updated from version 4.2.0 to 4.2.1 in the.github/workflows/release.yml
file. This update includes a new feature whererefs/*
are checked out by commit if provided, falling back to the ref specified by the@orhantoy
user. This change improves the flexibility of the action, allowing users to specify a commit or branch for checkout. The pull request also introduces a new contributor,@Jcambass
, who added a workflow file for publishing releases to an immutable action package. The commits for this release include changes to prepare for the 4.2.1 release, add a workflow file for publishing releases, and check out otherrefs/*
by commit if provided, falling back to ref. This pull request has been reviewed and approved by Dependabot.actions/checkout
dependency from version 4.2.1 to 4.2.2, which includes improvements to theurl-helper.ts
file that now utilize well-known environment variables and expanded unit test coverage for theisGhes
function. Theactions/checkout
action is commonly used in GitHub Actions workflows for checking out a repository at a specific commit or branch. The changes in this update are internal to theactions/checkout
action and should not affect the functionality of the project utilizing this action. The pull request also includes details on the commits and compatibility score for the upgrade, and reviewers can manage and merge the request using Dependabot commands once the changes have been verified.databrickslabs/sandbox
dependency has been updated from versionacceptance/v0.3.0
to0.3.1
. This update includes previously tagged commits, bug fixes for git-related libraries, and resolution of theunsupported protocol scheme
error. The README has been updated with more information on using thedatabricks labs sandbox
command, and installation instructions have been improved. Additionally, there have been dependency updates forgo-git
libraries andgolang.org/x/crypto
in the/go-libs
and/runtime-packages
directories. New commits in this release allow larger logs from acceptance tests and implement experimental OIDC refresh functionality. Ignore conditions have been applied to prevent conflicts with previous versions of the dependency. This update is recommended for users who want to take advantage of the latest bug fixes and improvements.databrickslabs/sandbox
dependency has been updated from versionacceptance/v0.3.1
to0.4.2
. This update includes bug fixes, dependency updates, and additional go-git libraries. Specifically, theRun integration tests
job in the GitHub Actions workflow has been updated to use the new version of thedatabrickslabs/sandbox/acceptance
Docker image. The updated version also includes install instructions, usage instructions in the README, and a modification to provide more git-related libraries. Additionally, there were several updates to dependencies, includinggolang.org/x/crypto
version0.16.0
to0.17.0
. Dependabot, a tool that manages dependencies in GitHub projects, is responsible for the update and provides instructions for resolving any conflicts or merging the changes into the project. This update is intended to improve the functionality and reliability of thedatabrickslabs/sandbox
dependency.Row.as_dict()
(#309). In this release, we are introducing a deprecation warning for theas_dict()
method in theRow
class, which will be removed in favor of theasDict()
method. This change aims to maintain consistency with Spark'sRow
behavior and prevent subtle bugs when switching between different backends. The deprecation warning will be implemented using Python's warnings mechanism, including the new annotation in Python 3.13 for static code analysis. The existing functionality of fetching values from the database throughStatementExecutionExt
remains unchanged. We recommend that clients update their code to use.asDict()
instead of.as_dict()
to avoid any disruptions. A new test casetest_row_as_dict_deprecated()
has been added to verify the deprecation warning forRow.as_dict()
..save_table(mode="overwrite")
(#298). In this release, the.save_table()
method has been improved, particularly when using theoverwrite
mode. If no rows are supplied, the table will now be truncated, ensuring consistency with the mock backend behavior. This change has been optimized for SQL-based backends, which now perform truncation as part of the insert for the first batch. Type hints on the abstract method have been updated to match the concrete implementations. Unit tests and integration tests have been updated to cover the new functionality, and new methods have been added to test the truncation behavior in overwrite mode. These improvements enhance the consistency and efficiency of the.save_table()
method when usingoverwrite
mode across different backends.databrickslabs/sandbox
package to versionacceptance/v0.3.0
in thedownstreams.yml
file. This update is necessary to use the latest version of the package, which includes several bug fixes and dependency updates. Thedatabrickslabs/sandbox
package is used in the acceptance tests, which are run as part of the CI/CD pipeline. It provides a set of tools and utilities for developing and testing code in a sandbox environment. The changelog for this version includes the addition of install instructions, more git-related libraries, and the modification of the README to include information about how to use it with thedatabricks labs sandbox
command. Specifically, the version of thedatabrickslabs/sandbox
package used in theacceptance
job has been updated fromacceptance/v0.1.4
toacceptance/v0.3.0
, allowing the integration tests to be run using the latest version of the package. The ignore conditions for this PR ensure that Dependabot will resolve any conflicts that may arise and can be manually triggered with the@dependabot rebase
command.Dependency updates: