search

quixdb / squash

Compression abstraction library and utilities
https://quixdb.github.io/squash/
MIT License
405 stars 53 forks source link

Some variadic functions rely on undefined behavior #224

Open nemequ opened 7 years ago

nemequ commented 7 years ago

clang 3.9+ picks up some potential undefined behavior I wasn't aware of: EXP58-CPP

Basically, anything subject to promotion can't be the last non-variadic argument in a variadic function. It's unlikely to be a problem, but technically enums aren't allowed. Need to make an API change to fix this, just not sure what the best option is yet…

Relevant functions are

SquashStream* squash_codec_create_stream (SquashCodec* codec, SquashStreamType stream_type, ...);
SquashStream* squash_stream_new (SquashCodec* codec, SquashStreamType stream_type, ...);

Possibly others, I haven't done a full check. One option would be reversing the stream_type and codec arguments, but that doesn't fit with the rest of the API. Changing SquashStreamType to an int would ruin -Wswitch-enum. Maybe drop SquashStreamType in favor of a bool…