search

databrickslabs / remorph

Cross-compiler and Data Reconciler into Databricks Lakehouse
Other
21 stars 13 forks source link

Snowflake: last mile to "full coverage" #544

Closed vil1 closed 1 month ago

vil1 commented 1 month ago

This PR fixes all (but one, see bellow) coverage tests. While being an important milestone, this by no means means that the Snowflake dialect is 100% supported, just that all the test queries we currently have at our disposal are properly parsed and translated to IR by the ANTLR-based transpiler.

This PR touches (lightly) many different areas of the implementation, in absolute, it should have been split in 5 or 6 (minute) separate PRs. But considering the current situation with the CI-review-auto-merge pipeline, I considered it worth cramming all these small changes into a single PR. Taking that into account, each commit on this branch tackles a specific defect, hence, reviewers are advised to look at this PR one commit at a time in order to make their life easier.

One test remains unfixed, namely tests/test_lateral_struct/test_lateral_struct_2.sql, but it showcases syntax (input => ${p}.${c}) that looks like classical string interpolation but isn't documented anywhere in Snowflake official docs... (maybe @vijaypavann-db could tell us more about this).

github-actions[bot] commented 1 month ago

Coverage tests results

394 tests  ±0   378 :white_check_mark: +11   4s :stopwatch: ±0s   2 suites ±0     0 :zzz: ± 0    2 files   ±0    16 :x:  - 11 

For more details on these failures, see this check.

Results for commit 43f9bc4f. ± Comparison against base commit cd0336bf.

:recycle: This comment has been updated with latest results.

sundarshankar89 commented 1 month ago

you can ignore this tests/test_lateral_struct/test_lateral_struct_2.sql as this was an effort convert dbt style queries.