search

Rust-SDL2 / rust-sdl2

SDL2 bindings for Rust
MIT License
2.65k stars 465 forks source link

bug: undefined behavior when transmuting u32 to various enums #1388

Open antonilol opened 2 months ago

antonilol commented 2 months ago

in several places in the code a u32 is transmuted to enums without any checks. examples include https://github.com/Rust-SDL2/rust-sdl2/blob/146cff67b05a7b9f20d126ee9f4891a5c2e5c881/src/sdl2/render.rs#L118, https://github.com/Rust-SDL2/rust-sdl2/blob/146cff67b05a7b9f20d126ee9f4891a5c2e5c881/src/sdl2/render.rs#L172 and https://github.com/Rust-SDL2/rust-sdl2/blob/146cff67b05a7b9f20d126ee9f4891a5c2e5c881/src/sdl2/pixels.rs#L448.

these TryInto implementations are public so this can be easily demonstrated by using miri with the code let b: BlendMode = 3.try_into() (this calls <BlendMode as TryFrom<u32>>::try_from(3) with the invalid value 3) for example:

error: Undefined Behavior: constructing invalid value at .<enum-tag>: encountered 0x00000003, but expected a valid enum tag
   --> /home/antoni/.cargo/registry/src/index.crates.io-6f17d22bba15001f/sdl2-0.36.0/src/sdl2/render.rs:172:27
    |
172 |         Ok(match unsafe { transmute(n) } {
    |                           ^^^^^^^^^^^^ constructing invalid value at .<enum-tag>: encountered 0x00000003, but expected a valid enum tag
    |
    = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
    = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
    = note: BACKTRACE:
    = note: inside `<sdl2::render::BlendMode as std::convert::TryFrom<u32>>::try_from` at /home/antoni/.cargo/registry/src/index.crates.io-6f17d22bba15001f/sdl2-0.36.0/src/sdl2/render.rs:172:27: 172:39
    = note: inside `<u32 as std::convert::TryInto<sdl2::render::BlendMode>>::try_into` at /home/antoni/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/convert/mod.rs:798:9: 798:26