alamb
commented
1 month ago Due Wed Sep 11 2024 (in 22 days)
Closed alamb closed 1 month ago
alamb
commented
1 month ago Due Wed Sep 11 2024 (in 22 days)
alamb
commented
1 month ago Draft document is https://docs.google.com/document/d/10MrrMHyfALB9Vpjtl-6abGWdNz6VLiWR9yI1a6Yl1aw/edit
I also sent a note to the mailing list as well, but it doesn't see it posted yet
alamb
commented
1 month ago This is due Wed Sep 11
According to our records, you are listed as the chair of DataFusion, a committee that is due to submit a report by Wed Sep 11th for the next ASF board meeting. This is an initial reminder to prepare a report for DataFusion and submit it as described below.
alamb
commented
1 month ago I updated the doc https://docs.google.com/document/d/10MrrMHyfALB9Vpjtl-6abGWdNz6VLiWR9yI1a6Yl1aw/edit and plan to submit in the 11th.
Current content for the curious:
The mission of Apache DataFusion is the creation and maintenance of software related to an extensible query engine
Current project status: New + Ongoing (high activity) Issues for the board: None
Apache DataFusion was founded 2024-04-16 (5 months ago) There are currently 37 committers and 14 PMC members in this project. The Committer-to-PMC ratio is roughly 5:2.
Community changes, past quarter:
The project continues to be active with many PRs and issues opened and closed per day.
We wrote two public blogs about our work: 1, 2 and DataFusion and systems built on it are being featured in high profile (for the Database world) venues such as the CMU Database Systems Seminar
We are working to adopt the sqlparser crate into the project as well
https://github.com/apache/datafusion
We continue the monthly release cadence versions 40.0.0, 41.0.0 and are on track for version 42.0.0. The 41.0.0 release had almost 70 unique contributors.
We are currently focused on performance including for high cardinality aggregates and adding support for StringViewArrays. We completed a long running project to ensure all aggregate functions use the same API and are beginning the same project for window functions.
We have been discussing what features to include, and working to add LogicalTypes, as well as to create a more differentiated CLI experience. See the roadmap ticket for more details.
https://github.com/apache/datafusion-python
The DataFusion Python project has received significant contributions recently to make the project more “Pythonic” and now has regular activity from maintainers. Tim Saucer has been added as a committer who focuses more heavily on datafusion-python.
https://github.com/apache/datafusion-comet
The Comet project is very active and recently released its initial 0.1.0 source release. Blog post: https://datafusion.apache.org/blog/2024/07/20/datafusion-comet-0.1.0/
https://github.com/apache/datafusion-ballista https://github.com/apache/datafusion-ballista-python
The Ballista subproject is not very actively maintained, but there have been some contributions recently to upgrade to more recent versions of the core DataFusion project.
It is still hard to keep track of everything going on these, which is a good thing. While it is always a struggle to get enough code review capacity, the committers keep things going and the community helps each other out with reviews. We continue to actively grow our committer and PMC ranks.
There are currently four meetups planned: New York City, San Francisco (for the second time!) Belgrade, and Seattle.
alamb
commented
1 month ago Final submitted report:
The mission of Apache DataFusion is the creation and maintenance of software related to an extensible query engine
Current project status: New + Ongoing (high activity) Issues for the board: None
Apache DataFusion was founded 2024-04-16 (5 months ago) There are currently 37 committers and 14 PMC members in this project. The Committer-to-PMC ratio is roughly 5:2.
Community changes, past quarter:
The project continues to be active with many PRs and issues opened and closed per day.
We wrote two public blogs about our work: 1, 2 and DataFusion and systems built on it are being featured in high profile (for the Database world) venues such as the CMU Database Systems Seminar
We are working to adopt the sqlparser crate into the project as well
https://github.com/apache/datafusion
We continue the monthly release cadence versions 40.0.0, 41.0.0 and are on track for version 42.0.0. The [41.0.0 release] had almost 70 unique contributors.
We are currently focused on performance including for high cardinality aggregates and adding support for StringViewArrays. We completed a long running project to ensure all aggregate functions use the same API and are beginning the same project for window functions.
We have been featuresg what features to include, and working to add LogicalTypes, as well as to create a more differentiated CLI experience. See the roadmap ticket for more details.
https://github.com/apache/datafusion-python
The DataFusion Python project has received significant contributions recently to make the project more “Pythonic” and now has regular activity from maintainers. Tim Saucer has been added as a committer who focuses more heavily on datafusion-python.
https://github.com/apache/datafusion-comet
The Comet project is very active and recently released its initial 0.1.0 source release.
Blog post: https://datafusion.apache.org/blog/2024/07/20/datafusion-comet-0.1.0/
https://github.com/apache/datafusion-ballista https://github.com/apache/datafusion-ballista-python
The Ballista subproject is not very actively maintained, but there have been some contributions recently to upgrade to more recent versions of the core DataFusion project.
It is still hard to keep track of everything going on these, which is a good thing. While it is always a struggle to get enough code review capacity, the committers keep things going and the community helps each other out with reviews. We continue to actively grow our committer and PMC ranks.
There are currently four meetups planned: New York City, San Francisco (for the second time!) Belgrade, and Seattle.
alamb
commented
1 month ago Next report is in december: https://github.com/apache/datafusion/issues/10157
Is your feature request related to a problem or challenge?
Per https://whimsy.apache.org/roster/committee/datafusion the DataFusion ASF board report schedule is
March, June, September, December
Describe the solution you'd like
I would like to draft a board report for the ASF board meeting, ideally with community help.
The meetings are typically in the second or third week of the month
Describe alternatives you've considered
I plan to do this in the same style that worked well in Arrow (see an example from @andygrove here https://lists.apache.org/thread/7w4mgy98qomc6drvj2fo81gvhq6p0boc) -- make a google doc (or issue) that people can add relevant content to and then the chair (me for the time being) submits it to the board
Additional context
No response