apache / arrow-nanoarrow

Helpers for Arrow C Data & Arrow C Stream interfaces
https://arrow.apache.org/nanoarrow
Apache License 2.0
173 stars 38 forks source link

Windows issue with negative decimals #594

Open WillAyd opened 2 months ago

WillAyd commented 2 months ago

Not sure if this is a nanoarrow issue or with the PyArrow export, but figured I'd start here.

When consuming an array like this:

    schema = pa.schema([("col", pa.decimal128(38, 10))])
    tbl = pa.Table.from_arrays([
        pa.array(["1234567890.123456789", "-99876543210.987654321",])
    ], schema=schema)

and using ArrowDecimalAppendDigitsToBuffer in nanoarrow, it seems like negative values on windows only are not properly represented.

I created an "MRE" for this in this library:

https://github.com/WillAyd/capsule_testing/tree/739bc608b96101ada424ef6d9736e2dfffcc2f11

You can see the issue for Windows in CI logs:

https://github.com/WillAyd/capsule_testing/actions/runs/10482665140/job/29034136906#step:4:358

paleolimbot commented 2 months ago

Thank you for the report and excellent example! I'll will take a look at this today or tomorrow.

WillAyd commented 2 months ago

Thanks @paleolimbot . To my surprise, I think this issue goes away when I use meson as the build system?

https://github.com/WillAyd/capsule_testing/pull/1/files

So may be something in the flags between CMake / Meson that causes this

WillAyd commented 2 months ago

If it helps, here is the verbose log for a CMake build:

https://github.com/WillAyd/capsule_testing/actions/runs/10494816933/job/29071897812?pr=3#step:4:286 https://github.com/WillAyd/capsule_testing/pull/3

```sh cl /c /I"C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-gs9hevtp\overlay\Lib\site-packages\nanobind\include" /I"C:\Users\runneradmin\AppData\Local\Temp\tmpn1av5dil\build\_deps\nanoarrow-project-src\src" /I"C:\Users\runneradmin\AppData\Local\Temp\tmpn1av5dil\build\_deps\nanoarrow-project-build\src" /W1 /WX- /diagnostics:column /MP /O2 /Ob2 /D _WINDLL /D _MBCS /D WIN32 /D _WINDOWS /D NDEBUG /D "CMAKE_INTDIR=\"Release\"" /D capsule_bug_EXPORTS /EHsc /MD /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /std:c++17 /Fo"capsule_bug.dir\Release\\" /Fd"capsule_bug.dir\Release\vc143.pdb" /external:W0 /Gd /TP /errorReport:queue /external:I "C:/Users/runneradmin/AppData/Local/pypa/cibuildwheel/Cache/nuget-cpython/python.3.9.13/tools/Include" /bigobj D:\a\capsule_testing\capsule_testing\src\capsule_bug\capsule_bug.cc ```

Versus the same thing in Meson:

https://github.com/WillAyd/capsule_testing/actions/runs/10494621884/job/29071248445?pr=1#step:4:382 https://github.com/WillAyd/capsule_testing/pull/1

```sh "c++" "-Icapsule_bug.cp39-win_amd64.pyd.p" "-I." "-I.." "-Isubprojects\arrow-nanoarrow-793590e45abfefa93f9fd1447b4b5d98266ccfca\src" "-I..\subprojects\arrow-nanoarrow-793590e45abfefa93f9fd1447b4b5d98266ccfca\src" "-I..\subprojects\nanobind-2.1.0\include" "-I..\subprojects\robin-map-1.3.0\include" "-IC:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\python.3.9.13\tools\Include" "-fvisibility=hidden" "-fvisibility-inlines-hidden" "-fdiagnostics-color=always" "-DNDEBUG" "-D_FILE_OFFSET_BITS=64" "-Wall" "-Winvalid-pch" "-Wextra" "-std=c++20" "-O3" "-DMS_WIN64=" -MD -MQ capsule_bug.cp39-win_amd64.pyd.p/src_capsule_bug_capsule_bug.cc.obj -MF "capsule_bug.cp39-win_amd64.pyd.p\src_capsule_bug_capsule_bug.cc.obj.d" -o capsule_bug.cp39-win_amd64.pyd.p/src_capsule_bug_capsule_bug.cc.obj "-c" ../src/capsule_bug/capsule_bug.cc ```