search

apache / arrow-nanoarrow

Helpers for Arrow C Data & Arrow C Stream interfaces
https://arrow.apache.org/nanoarrow
Apache License 2.0
169 stars 35 forks source link

Integration workflow is failing for Rust and nanoarrow #641

Open paleolimbot opened 1 day ago

paleolimbot commented 1 day ago

The Rust errors are due to https://github.com/apache/arrow-rs/issues/5052, and unfortunately I think it just means we have to disable checking Rust compatibility until the community has the bandwidth to review the fix ( https://github.com/apache/arrow-rs/pull/6426 ).

https://github.com/apache/arrow-nanoarrow/actions/runs/11130234810/job/30929145413#step:3:10644

  FAILED TEST: primitive Rust producing,  nanoarrow consuming
  <class 'subprocess.CalledProcessError'>: Command '/build/nanoarrow/nanoarrow_ipc_integration < /tmp/tmpyz3mdbbr/afddb6c7_generated_primitive.producer_file_as_stream' returned non-zero exit status 22.

  FAILED TEST: primitive_zerolength Rust producing,  nanoarrow consuming
  <class 'subprocess.CalledProcessError'>: Command '/build/nanoarrow/nanoarrow_ipc_integration < /tmp/tmpyz3mdbbr/018cdedf_generated_primitive_zerolength.producer_file_as_stream' returned non-zero exit status 22.

  FAILED TEST: primitive_large_offsets Rust producing,  nanoarrow consuming
  <class 'RuntimeError'>: Command failed: /build/nanoarrow/nanoarrow_ipc_integration
  With output:
  --------------
  Validating that /tmp/tmpyz3mdbbr/2dffa3bd_generated_primitive_large_offsets.json_as_file reads identical to /tmp/arrow-integration-8826mv00/generated_primitive_large_offsets.json
  Command VALIDATE failed (22=Invalid argument): Footer flatbuffer verification failed (16) vector header out of range or unaligned

  --------------

The nanoarrow--nanoarrow error is something about converting swapped endian IPC streams to IPC files since it seems like buffer endian swapping isn't taking place.

https://github.com/apache/arrow-nanoarrow/actions/runs/11130234810/job/30929145413#step:3:10810

FAILED TEST: interval nanoarrow producing,  nanoarrow consuming
  <class 'RuntimeError'>: Command failed: /build/nanoarrow/nanoarrow_ipc_integration
  With output:
  --------------
  Validating that /arrow-integration/testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_interval.arrow_file reads identical to /tmp/tmpyz3mdbbr/1.0.0-bigendian_interval.gold.json
  Command VALIDATE failed (22=Invalid argument): Found 12 differences between batches:
  Path: Batch 0.children[0]
  - {"name": "f1", "count": 7, "VALIDITY": [1, 1, 1, 1, 1, 1, 0], "DATA": ["128", "-129", "-5758134827132877538", "-5984203302733857979", "7152141364016051091", "1946707673701025977", "0"]}
  + {"name": "f1", "count": 7, "VALIDITY": [1, 1, 1, 1, 1, 1, 0], "DATA": ["-9223372036854775808", "9223372036854775807", "2199428923299600048", "4988814404219237292", "-7785723983210331805", "-5113702496382483429", "0"]}

  Path: Batch 0.children[1]
  - {"name": "f2", "count": 7, "VALIDITY": [1, 1, 0, 1, 1, 0, 1], "DATA": ["128", "-129", "0", "-8588564788114424009", "4714234549317537763", "0", "-915230144468162553"]}
  + {"name": "f2", "count": 7, "VALIDITY": [1, 1, 0, 1, 1, 0, 1], "DATA": ["-9223372036854775808", "9223372036854775807", "0", "4034946345919434632", "-2085181525232227263", "0", "569773930746957043"]}

  Path: Batch 0.children[2]
  - {"name": "f3", "count": 7, "VALIDITY": [0, 0, 1, 1, 1, 1, 1], "DATA": ["0", "0", "-91386228020645345", "8297967703166746363",