search

apache / arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics
https://arrow.apache.org/
Apache License 2.0
14.68k stars 3.56k forks source link

[C++] IPC: validate batch schema equals stream schema in debug mode #31977

Open asfimport opened 2 years ago

asfimport commented 2 years ago

This came up in a Flight/Flight SQL demo a colleague was working on; it was possible to write a batch with a differing schema than what was stated for the stream, which would lead to a decoding failure on the other side. It might be useful in DEBUG mode to DCHECK this and fail-fast.

The error message could also be improved; ArrayLoader.GetBuffer could at least return the index and the actual # of buffers

Reporter: David Li / @lidavidm

Note: This issue was originally created as ARROW-16625. Please see the migration documentation for further details.

lidavidm commented 3 weeks ago

See a writeup by @rustyconover in https://github.com/apache/arrow/issues/44689