search

apache / datafusion-comet

Apache DataFusion Comet Spark Accelerator
https://datafusion.apache.org/comet
Apache License 2.0
447 stars 100 forks source link

Implement Spark-compatible cast to/from binary type #377

Open andygrove opened 2 weeks ago

andygrove commented 2 weeks ago

What is the problem the feature request solves?

https://github.com/apache/datafusion-comet/pull/362 added some tests for casting to/from binary. Some pass and some are ignored because they currently fail.

This issue is for investigating the failures and either adding documentation to explain why we are incompatible, or making them compatible.

Describe the potential solution

No response

Additional context

No response

mattharder91 commented 2 weeks ago

I would like to work on this

andygrove commented 2 weeks ago

Thanks @mattharder91. Feel free to break this down into smaller issues if needed e.g. string <-> binary, integers <-> binary and so on.

There is a little more info on binary -> string in https://github.com/apache/datafusion-comet/pull/379. It looks this works for valid UTF-8 inputs but Spark allows non UTF-8 input.