Fails to compile on nightly some time after `2022-08-23`

[ralfbiedert]

To repeat what has been observed in #18 and others:

• The crate compiled fine with stable and nightly up until some time after 2022-08-23
• Some nightly after that apparently broke some unit tests
• So far it's unclear what went wrong, but the behavior is reproducible on Windows and Linux, locally and on CI.
• Solutions welcome!

---

From #18:

I just did some testing on my Linux machine, but the behavior is basically the same, it fails on +nightly in here

#[test]
fn can_decode_multi_by_step() -> Result<(), Error> {
    let src = &include_bytes!("data/multi_512x512.h264")[..];

    let config = DecoderConfig::default().debug(false);
    let mut decoder = Decoder::with_config(config)?;

    for packet in nal_units(src) {
        decoder.decode(packet)?; // <--- Commenting this line out makes this test pass.
    }

    Ok(())
}

Error:

---- can_decode_multi_by_step stdout ----
thread 'can_decode_multi_by_step' panicked at 'assertion failed: `(left == right)`
  left: `1`,
 right: `0`: the test returned a termination value with a non-zero status code (1) which indicates a failure', /rustc/95a992a68694d8bf3959bd2c0ac27ce9e9208b59/library/test/src/lib.rs:184:5
stack backtrace:
   0: rust_begin_unwind
             at /rustc/95a992a68694d8bf3959bd2c0ac27ce9e9208b59/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/95a992a68694d8bf3959bd2c0ac27ce9e9208b59/library/core/src/panicking.rs:142:14
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed
             at /rustc/95a992a68694d8bf3959bd2c0ac27ce9e9208b59/library/core/src/panicking.rs:181:5
   4: test::assert_test_result
             at /rustc/95a992a68694d8bf3959bd2c0ac27ce9e9208b59/library/test/src/lib.rs:184:5
   5: decode::can_decode_multi_by_step::{{closure}}
             at ./tests/decode.rs:64:1
   6: core::ops::function::FnOnce::call_once
             at /rustc/95a992a68694d8bf3959bd2c0ac27ce9e9208b59/library/core/src/ops/function.rs:248:5
   7: core::ops::function::FnOnce::call_once
             at /rustc/95a992a68694d8bf3959bd2c0ac27ce9e9208b59/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

[ralfbiedert]

This does not seem to be related to recent upstream changes. I just rolled back to 0712009 upstream and the error is still happens.

[ralfbiedert]

And interestingly enough:

• cargo +nightly test --features backtrace -- works
• cargo +nightly test -- fails

[ralfbiedert]

Something is really UB broken here:

Gives

[openh264\tests\decode.rs:71] 1 = 1
[openh264\src\decoder.rs:179] 2 = 2
[openh264\src\decoder.rs:188] 3 = 3
[openh264\src\decoder.rs:196] 4 = 4
[openh264\src\decoder.rs:198] "WILL RETURN OK" = "WILL RETURN OK"
[openh264\tests\decode.rs:73] &x = Err(
    Error {
        native: 0,
        decoding_state: 0,
        misc: None,
    },
)

In other words, Ok ends up as Err ...

[ralfbiedert]

Alright, the error was UB in here, where dst[x] were null. However, they should not be null, since OpenH264 did not return an error ...

            let y = std::slice::from_raw_parts(dst[0], (info.iHeight * info.iStride[0]) as usize);
            let u = std::slice::from_raw_parts(dst[1], (info.iHeight * info.iStride[1] / 2) as usize);
            let v = std::slice::from_raw_parts(dst[2], (info.iHeight * info.iStride[1] / 2) as usize);