substrait-io / substrait

A cross platform way to express data transformation, relational algebra, standardized record expression and plans.
https://substrait.io
Apache License 2.0
1.2k stars 158 forks source link

Proposal to donate BFT (Big Function Taxonomy) repo to Substrait project #658

Closed richtia closed 2 months ago

richtia commented 4 months ago

Hey Folks,

Over the last year, a few of us at Voltron Data have been working on a project to catalogue the functions defined in substrait and document their corner case behaviors when executed by different backend engines. One of the thought processes behind this was to make it easier to know which problems would be encountered when switching between these different backends. More details about the project can be seen in the README of the repository linked.

Repository: https://github.com/voltrondata/bft Site: https://voltrondata.github.io/bft/

This is definitely still a work in progress, but we feel like it's at a mature enough point to donate.

Here's the proposed sketch of a donation process laid out by @jacques-n in response to the initial email I sent out to the community mailing list.

1. We create an issue tracking this in the main Substrait repository.
2. SMC votes on acceptance of donation on that ticket. I suggest we need at least 3 +1s from SMC as well as no -1 from SMC. Any -1s from the broader community should also be thoroughly examined before moving forward.
3. A Voltron authorized person comments on the ticket stating that Voltron is agreeing to donate the code to Substrait community. (could come before 2 or after)
4. Voltron transfers the existing repository to the Substrait github organization.
jacques-n commented 4 months ago

I think this is a great idea. It was always a goal of substrait for us to build up this knowledge so here feels like the right home. @westonpace , @EpsilonPrime , @cpcloud and others, what's your thoughts?

EpsilonPrime commented 4 months ago

I'm in favor of accepting the donation. This will greatly complement the consumer testing work. Is the intention to keep the repository name the same?

richtia commented 4 months ago

We can keep the repository name the same or change it to something the community sees more fitting. I'm open to either.

richtia commented 4 months ago

@vbarua

westonpace commented 4 months ago

I'm in favor as well. I think the project goals are well aligned.

vbarua commented 4 months ago

I'm in favour of this as well.

cpcloud commented 4 months ago

+1!

jacques-n commented 3 months ago

Hey @richtia , checking on if you have an update. Would love to get this done!

richtia commented 3 months ago

Hey @richtia , checking on if you have an update. Would love to get this done!

Sorry for the delay. Just got back from PTO. We're working on drafting a comment for the grant/donation now! I have a PR to remove the substrait submodule from the BFT repo in order to help facilitate the donation and I'll add it back afterwards.

roaramburu commented 3 months ago

I am an authorized representative of Voltron Data, Inc. (the "Company"). The Company desires to contribute (the “Contribution”) the Big Function Taxonomy (BFT) software code repository located at https://github.com/voltrondata/bft (the "Repository") to the Substrait open source project (“Substrait”).

I represent that to the best of my knowledge, the Company has sufficient rights to make the Contribution, which is provided "AS IS", without warranty of any kind. The Company will not be liable for any damages of any kind arising in any way out of the use of the Repository and has no obligation to provide support, maintenance, updates, enhancements, modifications or ongoing warranties for the Repository.

Please confirm that you are an authorized representative of Substrait and that Substrait accepts the Contribution as set forth above.

jacques-n commented 3 months ago

Thanks @roaramburu, awesome to hear.

We accept the contribution given the SMC +1s above.

richtia commented 3 months ago

Transfer complete! https://github.com/substrait-io/bft

EpsilonPrime commented 2 months ago

Woohoo!