search

apache / arrow-rs

Official Rust implementation of Apache Arrow
https://arrow.apache.org/
Apache License 2.0
2.63k stars 803 forks source link

Brainstorm official support for arm32v7 hardware architecture #6681

Open drauschenbach opened 2 weeks ago

drauschenbach commented 2 weeks ago

This discussion aims to outline what would be required to add comprehensive support for the arm32v7 hardware architecture. Because it was in the top 3 best-selling computers of all time, and edge hardware is likely to remain in service for quite some time, and because edge is a particularly strong use case for Arrow.

And secondarily, consider how that might also benefit independent efforts to support WASM32.

Here are some of my own ideas:

  1. Perform tests and clippy static checks for both amd64 and armhf. amd64 must pass, armhf can fail until official support is completed.
  2. Tests that are specifically 64-bit in nature should not be attempted on 32-bit platforms (use #[cfg(target_pointer_width = "64")] around the test function). The POC at #6678 outlines most of these.
  3. Scrutinize test coverage for DecimalType, Decimal128Type and Decimal256Type which do not work on armhf.
  4. Scrutinize test coverage for Time64MicrosecondType and Time64NanosecondType, which do not work on armhf.
  5. Consider adding a badge to the README outlining current hardware platform support.
tustvold commented 2 weeks ago

I think there needs to be a proper assessment of APIs that currently use usize and would need to switch to u64, e.g. null counts, ArrowNativeType, etc... This would be a fairly disruptive and involved breaking change and so would require getting a group together to properly action this.

There are also a lot of APIs that assume performant u64, e.g. bitmap processing, I'm not sure if these would need to be changed.

Unfortunately I do not have the capacity to help drive this and am unlikely to for the foreseeable future