timescale / timescaledb-toolkit

Extension for more hyperfunctions, fully compatible with TimescaleDB and PostgreSQL 📈
https://www.timescale.com
Other
383 stars 47 forks source link

fix: correct unaligned slice access #827

Closed JamesGuthrie closed 1 week ago

JamesGuthrie commented 1 week ago

Tests started failing for an unknown reason. It turns out that we were attempting to build a slice with an unaligned pointer, and slice::from_raw_parts was panicking.

The reason the memory address is unaligned is that the Rust type has an 8-byte alignment requirement, but the Postgres type does not have a configured alignment, so it has the default 4-byte alignment. It is unclear why the type alignment was not configured in Postgres, presumably an oversight.

It appears as though we only end up with unaligned addresses when the types are stored on disk.

The affected code is doing zero-copy deserialization of Rust structs with Postgres-allocated memory. The deserialization code takes care of all internal alignment requirements, and assumes that the memory to be deserialized is aligned. Not all types are affected, only those which contain a slice of 8-byte aligned elements, because the slice is reconstructed with a (potentially 4-byte aligned) address.

This fix detects when we would attempt to deserialize data stored at an un-aligned address, and copies the data to a new (8-byte aligned) memory location, allowing for safe processing. It does not constrain this detection to only types with 8-byte aligned slices.

JamesGuthrie commented 1 week ago

Note: We should also follow up by setting the postgres type alignment to 'double' for all varlena types which require 8-byte alignment.