search

apache / drill

Apache Drill is a distributed MPP query layer for self describing data
https://drill.apache.org/
Apache License 2.0
1.93k stars 980 forks source link

DRILL-8412: Upgrade to Calcite 1.35-SNAPSHOT #2776

Closed jnturton closed 1 year ago

jnturton commented 1 year ago

DRILL-8412: Upgrade to Calcite 1.35-SNAPSHOT

Description

If we're willing to try basing Drill master on snapshot builds of the upcoming version of Calcite for a while then here's a PR to do that. Please see the related discussion in Drill and Calcite mailing lists this week for more information.

Important notes.

  1. The CI tests that run automatically upon commits to Drill master will exercise Drill with present day Calcite.
  2. Breaking changes in Calcite would (mostly) break the Drill CI and force us to deal with them in order to proceed.
  3. Regressions in Calcite would (mostly) break the Drill CI and force us to report them in order to proceed.
  4. If Drill master becomes too unstable when it is based on Calcite snapshots then this PR is trivially undoable.

Documentation

N/A

Testing

Existing unit test suite.

jnturton commented 1 year ago

@cgivre, @vvysotskyi, @rymarm, @luocooong are you up for running this experiment for a while? It's trivial to revert but I personally feel there's a good chance that we won't want to.

cgivre commented 1 year ago

@jnturton I think we should definitely run this experiment. I'm also curious as to how to run this with a specific PR from Calcite. That way I can contribute to the review process over there and find things that break drill quickly.

jnturton commented 1 year ago

@jnturton I think we should definitely run this experiment. I'm also curious as to how to run this with a specific PR from Calcite. That way I can contribute to the review process over there and find things that break drill quickly.

They won't build and publish artefacts for unmerged PRs (would be a security problem) so we'll have to run our own Calcite builds for these. Probably the best is for the developer to pull the PR into a local branch and build Calcite putting the results in their local Maven repo, then build Drill.

jnturton commented 1 year ago

Can someone do the formailty of approving so that we can give this a try?