jedisct1 / libsodium

A modern, portable, easy to use crypto library.
https://libsodium.org
Other
12.08k stars 1.72k forks source link

Fix build on z/OS using XLC #1089

Closed infinitydev closed 3 years ago

infinitydev commented 3 years ago

When compiling with IBMs XL C compiler, structure packing was done incorrectly in the BLAKE2 implementation. Packed structs must be enclosed by #pragma pack(1) and #pragma pack(pop) to push and pop the packing settings to/from the compiler's stack.

On AIX, #pragma pack() is equivalent to #pragma pack(pop) according to the docs: https://www.ibm.com/docs/en/xl-c-aix/13.1.2?topic=descriptions-pragma-pack

On z/OS, however, #pragma pack() is equivalent to #pragma pack(4) for C code: https://www.ibm.com/docs/en/zos/2.1.0?topic=descriptions-pragma-pack

So #pragma pack() will actually push to the stack instead of pop. As a result, when compiling for 64 bit, my own structs defined in a header were interpreted as packed with 4 byte alignment when used in a module that included my own header and sodium.h. In another module that did not include sodium.h, the struct was correctly interpreted as unpacked (i.e. 8 byte alignment). Passing my struct between both modules did not go well.

This is also why #pragma pack() is deprecated.