Take over maintenance/accept donation of DuckDB substrait-extension

[jacques-n]

I believe that there will be little development moving forward on Duckdb Substrait extension as a duckdb core extension. There have been no commits for a couple of months and the home page is fairly cool on accepting contributions "We do not have the capacity to engage with issues or pull requests from external contributors at present".

Myself and a few others have started enhancing the extension in a separate repo. However, we'd like to avoid having multiple active forks. I reached out to the DuckDB team to see if they would potentially be open to moving the main repo to the Substrait project so that we can continue it's development and growth. They've initially been receptive to this idea. I wanted to open discussion here to see whether others in the Substrait community are equally supportive of including accepting. Would love feedback from @substrait-io/smc as well as anyone else in the community. For example, I think @drin has also been working on this extension.

[drin]

Assuming the goal is to expand the number of users that can review and merge PRs by having the "primary" home be under substrait, then that seems reasonable to me.

[hannes]

DuckDB maintainer here, we are indeed happy with the idea of the substrait extension moving here.

[westonpace]

+1

Given the duckdb team is fine with transferring maintenance and you are comfortable with accepting maintenance I am having a hard time seeing any downsides. There may be some hiccups / compatibility differences between the two as people switch but long term compatibility between the projects seems well worth it. Thank you @jacques-n for taking this effort on.

[mbwhite]

+1 here; several months back I think I found a few things that needed tweaking.

[vbarua]

+1 from me as well. Would we want to set it up as a core repo, or a contrib repo?

[jacques-n]

Would we want to set it up as a core repo, or a contrib repo?

Either seems fine. I'm slightly inclined to go with non-contrib repo since the main challenge contrib is solving is ensuring access for non-substrait committers. I think at least four people who will be contributing are already committers so using contrib is probably less needed.

[drin]

the main challenge contrib is solving is ensuring access for non-substrait committers

Technical reason aside, I think it'd also be a bit weird to request to take ownership but then to call it a "contrib", so my preference would also be a core/standard repo

[jacques-n]

Seems like we have some positive energy here and 3 +1s from SMC (@westonpace , @vbarua , myself). Will leave open for another day or two for additional feedback before concluding. Would love feedback @EpsilonPrime and @cpcloud .

[EpsilonPrime]

As DuckDB is my favorite database and Substrait is my favorite cross-language relational algebra serializer I'm in favor of this proposal.

On Tue, Nov 5, 2024 at 7:51 PM Jacques Nadeau @.***> wrote:

Seems like we have some positive energy here and 3 +1s from SMC ( @westonpace https://github.com/westonpace , @vbarua https://github.com/vbarua , myself). Will leave open for another day or two for additional feedback before concluding. Would love feedback @EpsilonPrime https://github.com/EpsilonPrime and @cpcloud https://github.com/cpcloud .

— Reply to this email directly, view it on GitHub https://github.com/substrait-io/substrait/issues/731#issuecomment-2458673848, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABDDCBMF323PVANZNTTJNDZ7GG5TAVCNFSM6AAAAABRFTJKQ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJYGY3TGOBUHA . You are receiving this because you were mentioned.Message ID: @.***>

[ianmcook]

+1 from me. Thank you!

[jacques-n]

Seems like all positive here, lets move forward. Will post here as transfer is completed.

[jacques-n]

Repo has been moved over. 🎉 Thanks to everyone, especially the DuckDB team and @hannes !

https://github.com/substrait-io/duckdb-substrait-extension