apache datafusion-comet issues

apache / datafusion-comet

Apache DataFusion Comet Spark Accelerator

https://datafusion.apache.org/comet

Apache License 2.0

823 stars 163 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

ScanExec metrics do not get reported in Spark UI for aggregate, join, sort, etc

#1110 andygrove opened 8 hours ago
0
Native metrics do not always get mapped to the correct Spark nodes

#1109 andygrove opened 9 hours ago
0
chore: Add custom metric for native shuffle fetching batches from JVM

#1108 andygrove closed 7 hours ago
2
We do not record metrics for exchange read time

#1107 andygrove closed 1 day ago
0
fix: Support partition values in feature branch comet-parquet-exec

#1106 viirya opened 1 day ago
0
chore: Include first ScanExec batch in metrics

#1105 andygrove closed 2 days ago
0
feat: Use global tokio runtime per executor

#1104 andygrove opened 2 days ago
1
[comet-parquet-exec] Use Parquet schema for scan instead of Spark schema

#1103 mbutrovich opened 2 days ago
0
Support partition values in feature branch comet-parquet-exec

#1102 viirya opened 2 days ago
0
perf: Stop passing Java config map into native createPlan

#1101 andygrove opened 2 days ago
1
chore: Improve CometScan metrics

#1100 andygrove closed 1 day ago
4
feat: Add ability to save protobuf plans to disk for debugging

#1099 andygrove opened 2 days ago
1
Investigate native query planning overhead

#1098 andygrove opened 3 days ago
5
perf: Stop performing deep copies when CometScan source is an exchange

#1097 andygrove opened 3 days ago
0
docs: Update documentation for 0.4.0 release

#1096 andygrove closed 3 days ago
0
fix: Unsigned type related bugs

#1095 kazuyukitanimura closed 3 days ago
1
feat: Hook DataFusion Parquet native scan with Comet execution

#1094 viirya closed 2 days ago
4
minor: Add hint for finding the GPG key to use when publishing to maven

#1093 andygrove closed 6 days ago
0
Missing ColumnarToRow when using CometSparkToColumnar

#1092 bmorck opened 6 days ago
7
build: Skip installation of spark-integration and fuzz testing modules

#1091 parthchandra closed 6 days ago
1
chore: Prepare for 0.5.0 development

#1090 andygrove closed 6 days ago
0
chore: Add changelog for 0.4.0

#1089 andygrove closed 6 days ago
0
[comet-parquet-exec] Handle CometNativeScan RDD when DataSourceRDD instead of FileScanRDD

#1088 mbutrovich closed 6 days ago
0
test: Restore one test in CometExecSuite by adding COMET_SHUFFLE_MODE config

#1087 viirya closed 1 week ago
1
feat: enable decimal to decimal cast of different precision and scale

#1086 himadripal opened 1 week ago
2
docs: Update benchmarking.md

#1085 rluvaton-flarion closed 1 week ago
1
with datafusion comet，no performance improvement.

#1084 shaileneF opened 1 week ago
5
[comet-parquet-exec] update some stability plans

#1083 andygrove closed 1 week ago
0
Add Parquet column index support

#1082 kazuyukitanimura opened 1 week ago
0
[comet-parquet-exec] Pass Spark's partitions to DF's ParquetExec

#1081 mbutrovich closed 1 week ago
0
[comet-parquet-exec] Fix compilation errors in Rust tests, remove some debug logging

#1080 andygrove closed 1 week ago
0
Provide pip install for PySpark users

#1079 andygrove opened 1 week ago
3
wip - CometNativeScan

#1078 parthchandra closed 1 week ago
0
Native PhysicalPlanner extension points

#1077 mwiewior opened 1 week ago
1
wip - CometNativeScan

#1076 parthchandra closed 1 week ago
0
[WIP] [comet-parquet-exec] Add unit test for reading a struct field from Parquet

#1075 andygrove opened 1 week ago
0
feat: Implement CAST between struct types

#1074 andygrove closed 1 week ago
2
feat: support array_insert

#1073 SemyonSinchenko opened 1 week ago
8
feat: support array_append

#1072 NoeB closed 1 week ago
5
Comet 0.4.0 Release

#1071 andygrove closed 9 hours ago
0
feat: Use official DataFusion 43 release

#1070 andygrove closed 1 week ago
0
chore: Refactor binary and math expression serde code

#1069 andygrove closed 1 week ago
1
Make decimal to string cast fully compatible with Spark

#1068 andygrove opened 1 week ago
0
Incorrect behavior when reading unsigned integers from Parquet

#1067 andygrove closed 3 days ago
1
feat: Implement CAST from struct to string

#1066 andygrove closed 1 week ago
1
minor: refactor UnaryExpr handling to make code more concise

#1065 andygrove closed 1 week ago
1
CometShuffleMemoryAllocator should use Spark unified memory manager

#1064 viirya closed 1 week ago
0
chore: Simplify CometShuffleMemoryAllocator to use Spark unified memory allocator

#1063 viirya closed 1 week ago
26
feat: Require offHeap memory to be enabled (always use unified memory)

#1062 andygrove closed 1 week ago
5
WIP: Create separate instance of CometShuffleMemoryAllocator per plan

#1061 andygrove closed 2 weeks ago
1