pgcentralfoundation / pgrx

Build Postgres Extensions with Rust!
Other
3.7k stars 249 forks source link

FlatArray and Text #1856

Open workingjubilee opened 2 months ago

workingjubilee commented 2 months ago

Introduce a new Rust type that serves as a definition of what Postgres ArrayTypes "really are", called FlatArray. This is meant to be manipulated via &FlatArray<'_, T>... it is an "unsized type". It uses BorrowDatum to power its simplicity in iteration, compared to the incredibly jank implementation in pgrx/src/datum/array.rs! I should know it's jank, because I wrote it!

The cost is that it is much more nitpicky about what it can work with, as for a given FlatArray<'_, T>, we also require BorrowDatum in order to do anything useful with it! This means FlatArray can't be, say, FlatArray<'mcx, String>, and implicitly unbox elements into allocated types. You can still just use .iter().map(), of course, and the good news is this means all FlatArrays must truly be "zero-copy", as the type definition doesn't allow anything else.

As part of this compromise, we introduce a new Text type so that it is still possible to work with common array needs like strings.

Together, these are effectively the first varlena types that allow directly interacting with the data in their varlena header instead of "forgetting" it.

workingjubilee commented 1 month ago

This needs more tests, and I'll probably split it into components so that it can be reviewed in smaller chunks, but I'm satisfied with the fundamentals of the implementation here.

workingjubilee commented 1 month ago

Details of Text's impl are unsound (or at least, hard to use soundly?). I am going to PR it separately with sufficient tests.

workingjubilee commented 1 month ago

I think it mostly needs to be as pedantic about alignment as I was afraid of.