Open weiting-chen opened 6 months ago
@weiting-chen, for item
Do you know what changes to expect for Delta?
thanks
@weiting-chen, for item
- Data Lake Support([VL] Unified design for data lake read support in Gluten + Velox #3378 ): DeltaLake, Iceberg, Hudi, Paimon
Do you know what changes to expect for Delta?
thanks
The plan is to support both COW table and MOR table this year, which means DeltaLake with and without Deletion Vector support.
Thanks @weiting-chen
By Delta Deletion Vector support you mean for reads and writes?
Hello @weiting-chen,
Thanks a lot for concluding the roadmap.
Regarding supporting Spark 3.5.1 in general, I saw there are two PRs merged.
https://github.com/apache/incubator-gluten/pull/4822 https://github.com/apache/incubator-gluten/pull/4845
May I ask there are any other works to be done in order to support Spark 3.5.1 on the plan, or these MRs concludes the work to support Spark 3.5.1?
Thanks @weiting-chen for sharing the roadmap.
Regarding the item Other Big Data Framework Support
, are there any designs or action items that could be shared here?
Additionally, I saw an existing but archived effort at https://github.com/oap-project/Gluten-Trino, could you please provide more details or share your thoughts about it?
Description
Here listed Gluten roadmap in 2024. We will use this list to have an overview for Gluten, an easy way to track feature support plan, and to avoid duplicate work before starting the implementation.