Closed galipremsagar closed 2 months ago
@bdice I found the PR that was responsible for resolving this issue in the arrow project: https://github.com/apache/arrow/pull/35997
Would you be able to take a look at this in libcudf
?
I'm hoping to work on this in 24.02 but not sure if I'll have time in this release. It's on my list of tasks.
I'm working on a set of changes to try to address this.
Update: This work has been broken into many separate PRs, 4 of which have been merged so far:
Explicit conversion PR Benchmarking PR Powers-of-10 PR Utilities PR
The primary PR will be submitted very soon. I'm putting some finishing touches on it based off of conversations with @ttnghia on handling some of the edge cases.
There will always be discrepancies between cuDF and pyarrow, because cuDF has chosen to truncate its results, whereas pyarrow has chosen to round them.
The new decimal <--> floating conversion PR has been merged.
Thanks @pmattione-nvidia ! Closing this issue, was able to verify that the issue is fixed.
Describe the bug When there is a type-cast from
float32
toDecimal64Dtype
, the decimal values yield incorrect results. This issue was surfaced when testingarrow-13
.arrow-12
has the same bug and it was fixed inarrow-13
, we have similar issue in our libcudf too.Steps/Code to reproduce bug
Expected behavior
Environment overview (please complete the following information)
Environment details Please run and paste the output of the
cudf/print_env.sh
script here, to gather any other relevant environment detailsClick here to see environment details
Additional context Add any other context about the problem here.