voltrondata / substrait-r

An R Interface to the 'Substrait' Cross-Language Serialization for Relational Algebra
Other
27 stars 7 forks source link

dplyr::arrange() doesn't work in Arrow #68

Open thisisnic opened 2 years ago

thisisnic commented 2 years ago

This may be just due to it not being wired up in the Arrow Compute Engine yet.

library(substrait, warn.conflicts = FALSE)
library(dplyr, warn.conflicts = FALSE)

df <- data.frame(number = 1:5, letter = letters[1:5])

compiler <- df %>% 
  arrow_substrait_compiler() %>%
  arrange(desc(number))

collect(compiler)
#> Error: NotImplemented: conversion to arrow::compute::Declaration from Substrait relation sort {
#>   input {
#>     read {
#>       base_schema {
#>         names: "number"
#>         names: "letter"
#>         struct {
#>           types {
#>             i32 {
#>             }
#>           }
#>           types {
#>             string {
#>             }
#>           }
#>         }
#>       }
#>       local_files {
#>         items {
#>           uri_file: "file:///var/folders/gt/l87wjg8s7312zs9s7c1fgs900000gn/T//RtmpfPLtzY/filee26c74ef271b"
#>           format: FILE_FORMAT_PARQUET
#>         }
#>       }
#>     }
#>   }
#>   sorts {
#>     expr {
#>       selection {
#>         direct_reference {
#>           struct_field {
#>           }
#>         }
#>       }
#>     }
#>     direction: SORT_DIRECTION_DESC_NULLS_LAST
#>   }
#> }
#> 
#> /Users/deweydunnington/Desktop/rscratch/arrow/cpp/src/arrow/engine/substrait/serde.cc:73  FromProto(plan_rel.rel(), ext_set)

Created on 2022-04-11 by the reprex package (v2.0.1)

thisisnic commented 2 years ago

This is being tracked in ARROW-16649