search

xiph / rav1e

The fastest and safest AV1 encoder.
BSD 2-Clause "Simplified" License
3.72k stars 253 forks source link

Crash on encode fuzztest #3216

Closed tmatth closed 1 year ago

tmatth commented 1 year ago

Describe the bug I discovered this crash on fuzz testing encode:


Failing input:

    fuzz/artifacts/encode/crash-6c4a10dc08c47e5e5e818287d7dc742b58548506

Output of `std::fmt::Debug`:

    ArbitraryEncoder {
        config: Config {
            enc: EncoderConfig {
                width: 247,
                height: 80,
                sample_aspect_ratio: Rational {
                    num: 5714872554696429391,
                    den: 5714874072313712463,
                },
                time_base: Rational {
                    num: 8448,
                    den: 620756992,
                },
                bit_depth: 8,
                chroma_sampling: Cs420,
                chroma_sample_position: Unknown,
                pixel_range: Limited,
                color_description: None,
                mastering_display: None,
                content_light: None,
                level_idx: Some(
                    31,
                ),
                enable_timing_info: false,
                still_picture: true,
                error_resilient: false,
                switch_frame_interval: 0,
                min_key_frame_interval: 0,
                max_key_frame_interval: 1,
                reservoir_frame_delay: None,
                low_latency: true,
                quantizer: 1,
                min_quantizer: 43,
                bitrate: 16384,
                tune: Psychovisual,
                film_grain_params: None,
                tile_cols: 0,
                tile_rows: 0,
                tiles: 0,
                speed_settings: SpeedSettings {
                    multiref: false,
                    fast_deblock: true,
                    rdo_lookahead_frames: 10,
                    scene_detection_mode: Standard,
                    cdef: true,
                    lrf: false,
                    lru_on_skip: false,
                    sgr_complexity: Reduced,
                    segmentation: Simple,
                    partition: PartitionSpeedSettings {
                        encode_bottomup: false,
                        non_square_partition_max_threshold: BLOCK_8X8,
                        partition_range: PartitionRange {
                            min: BLOCK_16X16,
                            max: BLOCK_32X32,
                        },
                    },
                    transform: TransformSpeedSettings {
                        reduced_tx_set: true,
                        tx_domain_distortion: true,
                        tx_domain_rate: false,
                        rdo_tx_decision: false,
                        enable_inter_tx_split: true,
                    },
                    prediction: PredictionSpeedSettings {
                        prediction_modes: Simple,
                        fine_directional_intra: true,
                    },
                    motion: MotionSpeedSettings {
                        use_satd_subpel: true,
                        include_near_mvs: false,
                        me_allow_full_search: false,
                    },
                },
            },
            rate_control: RateControlConfig {
                emit_pass_data: false,
                summary: None,
            },
            threads: 1,
            pool: None,
        },
        frame_count: 1,
        pixels: [
            79,
            79,
            81,
            79,
            79,
            255,
            255,
            79,
            79,
            79,
        ],
    }

Reproduce with:

    cargo fuzz run encode fuzz/artifacts/encode/crash-6c4a10dc08c47e5e5e818287d7dc742b58548506

Minimize test case with:

    cargo fuzz tmin encode fuzz/artifacts/encode/crash-6c4a10dc08c47e5e5e818287d7dc742b58548506

To Reproduce Steps to reproduce the behavior cargo fuzz run encode fuzz/artifacts/encode/crash-6c4a10dc08c47e5e5e818287d7dc742b58548506

Expected behavior No crash

Required Information Toolchain (if is a build problem):

cargo 1.72.0-nightly (64fb38c97 2023-05-23)
rustc 1.72.0-nightly (498553fc0 2023-05-29)
NASM version 2.16.01

Version:

$ rav1e --version

Operating system:

$ Linux bellini 6.2.0-20-generic #20-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr  6 07:48:48 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Crash artifact: crash.zip

tmatth commented 1 year ago

Backtrace:

tmatth@bellini:/big-repos/rav1e$ RUST_BACKTRACE=full cargo fuzz run encode fuzz/artifacts/encode/crash-6c4a10dc08c47e5e5e818287d7dc742b58548506
    Finished release [optimized] target(s) in 0.09s
    Finished release [optimized] target(s) in 0.08s
     Running `fuzz/target/x86_64-unknown-linux-gnu/release/encode -artifact_prefix=/big-repos/rav1e/fuzz/artifacts/encode/ fuzz/artifacts/encode/crash-6c4a10dc08c47e5e5e818287d7dc742b58548506`
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3315938802
INFO: Loaded 1 modules   (585198 inline 8-bit counters): 585198 [0x556212c0b990, 0x556212c9a77e), 
INFO: Loaded 1 PC tables (585198 PCs): 585198 [0x556212c9a780,0x556213588660), 
fuzz/target/x86_64-unknown-linux-gnu/release/encode: Running 1 inputs 1 time(s) each.
Running: fuzz/artifacts/encode/crash-6c4a10dc08c47e5e5e818287d7dc742b58548506
thread '<unnamed>' panicked at 'assertion failed: min <= max', /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/core/src/cmp.rs:840:9
stack backtrace:
   0:     0x55621260b121 - std::backtrace_rs::backtrace::libunwind::trace::h49c88cf40170b5d6
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0x55621260b121 - std::backtrace_rs::backtrace::trace_unsynchronized::hdc6cc34fd8a53678
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x55621260b121 - std::sys_common::backtrace::_print_fmt::h0c7289f3a8de0b23
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/std/src/sys_common/backtrace.rs:65:5
   3:     0x55621260b121 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hc8e96ca3c7ab0680
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/std/src/sys_common/backtrace.rs:44:22
   4:     0x55621266969f - core::fmt::rt::Argument::fmt::h304ace7fe8bb186b
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/core/src/fmt/rt.rs:138:9
   5:     0x55621266969f - core::fmt::write::ha469f5b722d475f3
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/core/src/fmt/mod.rs:1094:21
   6:     0x5562125ffb71 - std::io::Write::write_fmt::h9a4844905af525b3
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/std/src/io/mod.rs:1713:15
   7:     0x55621260af35 - std::sys_common::backtrace::_print::ha35f1d4a0d8accc0
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/std/src/sys_common/backtrace.rs:47:5
   8:     0x55621260af35 - std::sys_common::backtrace::print::h690d5e347a585903
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/std/src/sys_common/backtrace.rs:34:9
   9:     0x55621260dbf7 - std::panicking::default_hook::{{closure}}::h819f1ab5f2b6d32d
  10:     0x55621260d9e4 - std::panicking::default_hook::hcdf96a30790dc6bb
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/std/src/panicking.rs:288:9
  11:     0x5562125513ca - libfuzzer_sys::initialize::{{closure}}::h645bf9c93deae018
  12:     0x55621260e2cd - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h4d61e66e458ff2d1
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/alloc/src/boxed.rs:1999:9
  13:     0x55621260e2cd - std::panicking::rust_panic_with_hook::hb101d99ddd165575
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/std/src/panicking.rs:709:13
  14:     0x55621260e021 - std::panicking::begin_panic_handler::{{closure}}::h6826cb5e0c9d0ead
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/std/src/panicking.rs:595:13
  15:     0x55621260b556 - std::sys_common::backtrace::__rust_end_short_backtrace::h7a916e155aec8921
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/std/src/sys_common/backtrace.rs:151:18
  16:     0x55621260ddb2 - rust_begin_unwind
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/std/src/panicking.rs:593:5
  17:     0x55621097b933 - core::panicking::panic_fmt::h1e09a982e3a0f5bf
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/core/src/panicking.rs:67:14
  18:     0x55621097b9c3 - core::panicking::panic::h6095d4fd94a5d06f
                               at /rustc/498553fc04f6a3fdc53412320f4e913bc53bc267/library/core/src/panicking.rs:117:5
  19:     0x556211a685b8 - rav1e::tiling::tiler::TilingInfo::from_target_tiles::h0ad5e313967e0724
  20:     0x556211978943 - rav1e::encoder::Sequence::new::h7a3c61375e3198d9
  21:     0x556211a9335b - rav1e::api::internal::ContextInner<T>::new::h404dbf653bd471e1
  22:     0x556211a7a896 - rav1e::api::config::Config::new_inner::h96199cb209809802
  23:     0x556211a830f4 - rav1e::api::config::Config::new_context::h4ac3a1529aad38f6
  24:     0x556211b24079 - rav1e::fuzzing::fuzz_encode::h9d528d0b531ae87c
  25:     0x556210a6d1a7 - encode::_::__libfuzzer_sys_run::hb9036b8c632549c8
  26:     0x556210a6c356 - rust_fuzzer_test_input
  27:     0x55621254c499 - std::panicking::try::do_call::hbdd3d3df20b314ce
  28:     0x5562125515e8 - __rust_try
  29:     0x5562125509a6 - LLVMFuzzerTestOneInput
  30:     0x55621255776e - _ZN6fuzzer6Fuzzer15ExecuteCallbackEPKhm
  31:     0x556212566712 - _ZN6fuzzer10RunOneTestEPNS_6FuzzerEPKcm
  32:     0x55621256ea4f - _ZN6fuzzer12FuzzerDriverEPiPPPcPFiPKhmE
  33:     0x55621097c397 - main
  34:     0x7fe226623a90 - __libc_start_call_main
                               at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
  35:     0x7fe226623b49 - __libc_start_main_impl
                               at ./csu/../csu/libc-start.c:360:3
  36:     0x55621097c3e5 - _start
  37:                0x0 - <unknown>
==2162533== ERROR: libFuzzer: deadly signal
    #0 0x556210a1e4a1  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x10274a1) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #1 0x556212591b2d  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2b9ab2d) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #2 0x556212557229  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2b60229) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #3 0x7fe22663c4af  (/lib/x86_64-linux-gnu/libc.so.6+0x3c4af) (BuildId: bdb8aa3b1b60f9d43e1c70ba98158e05f765efdc)
    #4 0x7fe226690ffa  (/lib/x86_64-linux-gnu/libc.so.6+0x90ffa) (BuildId: bdb8aa3b1b60f9d43e1c70ba98158e05f765efdc)
    #5 0x7fe22663c405  (/lib/x86_64-linux-gnu/libc.so.6+0x3c405) (BuildId: bdb8aa3b1b60f9d43e1c70ba98158e05f765efdc)
    #6 0x7fe22662287b  (/lib/x86_64-linux-gnu/libc.so.6+0x2287b) (BuildId: bdb8aa3b1b60f9d43e1c70ba98158e05f765efdc)
    #7 0x556212619ca6  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2c22ca6) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #8 0x556210978236  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0xf81236) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #9 0x5562125513d4  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2b5a3d4) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #10 0x55621260e2cc  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2c172cc) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #11 0x55621260e020  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2c17020) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #12 0x55621260b555  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2c14555) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #13 0x55621260ddb1  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2c16db1) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #14 0x55621097b932  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0xf84932) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #15 0x55621097b9c2  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0xf849c2) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #16 0x556211a685b7  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x20715b7) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #17 0x556211978942  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x1f81942) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #18 0x556211a9335a  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x209c35a) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #19 0x556211a7a895  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2083895) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #20 0x556211a830f3  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x208c0f3) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #21 0x556211b24078  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x212d078) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #22 0x556210a6d1a6  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x10761a6) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #23 0x556210a6c355  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x1075355) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #24 0x55621254c498  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2b55498) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #25 0x5562125515e7  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2b5a5e7) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #26 0x5562125509a5  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2b599a5) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #27 0x55621255776d  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2b6076d) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #28 0x556212566711  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2b6f711) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #29 0x55621256ea4e  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0x2b77a4e) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #30 0x55621097c396  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0xf85396) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)
    #31 0x7fe226623a8f  (/lib/x86_64-linux-gnu/libc.so.6+0x23a8f) (BuildId: bdb8aa3b1b60f9d43e1c70ba98158e05f765efdc)
    #32 0x7fe226623b48  (/lib/x86_64-linux-gnu/libc.so.6+0x23b48) (BuildId: bdb8aa3b1b60f9d43e1c70ba98158e05f765efdc)
    #33 0x55621097c3e4  (/big-repos/rav1e/fuzz/target/x86_64-unknown-linux-gnu/release/encode+0xf853e4) (BuildId: 0606e8d8508286eb2c5e43fa35e790f19c286b70)

NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
────────────────────────────────────────────────────────────────────────────────

Error: Fuzz target exited with exit status: 77
barrbrain commented 1 year ago 
barrbrain@battleship:~/rav1e$ RUST_BACKTRACE=full rustup run nightly cargo fuzz run -D encode fuzz/artifacts/encode/crash-6c4a10dc08c47e5e5e818287d7dc742b58548506
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `fuzz/target/x86_64-unknown-linux-gnu/debug/encode -artifact_prefix=/home/barrbrain/rav1e/fuzz/artifacts/encode/ fuzz/artifacts/encode/crash-6c4a10dc08c47e5e5e818287d7dc742b58548506`
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3771258067
INFO: Loaded 1 modules   (1035247 inline 8-bit counters): 1035247 [0x55fa83ca36d0, 0x55fa83da02bf), 
INFO: Loaded 1 PC tables (1035247 PCs): 1035247 [0x55fa83da02c0,0x55fa84d6c1b0), 
fuzz/target/x86_64-unknown-linux-gnu/debug/encode: Running 1 inputs 1 time(s) each.
Running: fuzz/artifacts/encode/crash-6c4a10dc08c47e5e5e818287d7dc742b58548506
thread '<unnamed>' panicked at 'assertion failed: min <= max', /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/core/src/cmp.rs:840:9
stack backtrace:
   0:     0x55fa833483d1 - std::backtrace_rs::backtrace::libunwind::trace::hc317256c0daecb22
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0x55fa833483d1 - std::backtrace_rs::backtrace::trace_unsynchronized::h85008105ccd95ba6
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x55fa833483d1 - std::sys_common::backtrace::_print_fmt::hb6e7cbe31fecc91c
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/sys_common/backtrace.rs:65:5
   3:     0x55fa833483d1 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h53097d3751045778
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/sys_common/backtrace.rs:44:22
   4:     0x55fa833a699f - core::fmt::rt::Argument::fmt::hdbe0b5dfb2c09e86
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/core/src/fmt/rt.rs:138:9
   5:     0x55fa833a699f - core::fmt::write::h4ae17c68778671a0
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/core/src/fmt/mod.rs:1094:21
   6:     0x55fa8333cec1 - std::io::Write::write_fmt::hfd7c55e775c6dc60
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/io/mod.rs:1713:15
   7:     0x55fa833481e5 - std::sys_common::backtrace::_print::h81eabb0036b692a0
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/sys_common/backtrace.rs:47:5
   8:     0x55fa833481e5 - std::sys_common::backtrace::print::h1b1f4d1a23fac3fd
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/sys_common/backtrace.rs:34:9
   9:     0x55fa8334aea7 - std::panicking::default_hook::{{closure}}::hc17d2bd8f3da442c
  10:     0x55fa8334ac94 - std::panicking::default_hook::h3a32a9336113ba96
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/panicking.rs:288:9
  11:     0x55fa831f9620 - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h283cf2930052d01c
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/alloc/src/boxed.rs:1999:9
  12:     0x55fa831f80ef - libfuzzer_sys::initialize::{{closure}}::h456beef92a984387
                               at /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/src/lib.rs:90:9
  13:     0x55fa8334b57d - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h4413820818220cb9
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/alloc/src/boxed.rs:1999:9
  14:     0x55fa8334b57d - std::panicking::rust_panic_with_hook::h90e4d15277396259
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/panicking.rs:709:13
  15:     0x55fa8334b2d1 - std::panicking::begin_panic_handler::{{closure}}::h2b743ea215b7c408
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/panicking.rs:595:13
  16:     0x55fa83348806 - std::sys_common::backtrace::__rust_end_short_backtrace::h895d390908a90650
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/sys_common/backtrace.rs:151:18
  17:     0x55fa8334b062 - rust_begin_unwind
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/panicking.rs:593:5
  18:     0x55fa7e38cc63 - core::panicking::panic_fmt::h1cc8517ae3119d0e
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/core/src/panicking.rs:67:14
  19:     0x55fa7e38ccf3 - core::panicking::panic::h8b995885e3e255d1
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/core/src/panicking.rs:117:5
  20:     0x55fa80e8bb83 - core::cmp::Ord::clamp::ha0286db42bbb9b9b
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/core/src/cmp.rs:840:9
  21:     0x55fa80ba7a03 - rav1e::tiling::tiler::TilingInfo::from_target_tiles::hc6d98d065944c005
                               at /home/barrbrain/rav1e/src/tiling/tiler.rs:135:26
  22:     0x55fa8073e392 - rav1e::encoder::Sequence::new::h1b3e43e4b23a5c40
                               at /home/barrbrain/rav1e/src/encoder.rs:240:22
  23:     0x55fa7ffd44de - rav1e::api::internal::ContextInner<T>::new::h133d8f040e7a27ef
                               at /home/barrbrain/rav1e/src/api/internal.rs:275:24
  24:     0x55fa7fee5e73 - rav1e::api::config::Config::new_inner::hbce573dbda0a7e6d
                               at /home/barrbrain/rav1e/src/api/config/mod.rs:241:21
  25:     0x55fa7fee806c - rav1e::api::config::Config::new_context::h801a83d55a2dd069
                               at /home/barrbrain/rav1e/src/api/config/mod.rs:293:17
  26:     0x55fa7fa27970 - rav1e::fuzzing::fuzz_encode::h323954c405063201
                               at /home/barrbrain/rav1e/src/fuzzing.rs:275:13
  27:     0x55fa7e463b86 - encode::_::__libfuzzer_sys_run::heebaf14ed9d99478
                               at /home/barrbrain/rav1e/fuzz/fuzz_targets/encode.rs:18:3
  28:     0x55fa7e463497 - rust_fuzzer_test_input
                               at /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/src/lib.rs:297:60
  29:     0x55fa831f6fbb - libfuzzer_sys::test_input_wrap::{{closure}}::h1a2970ccf079a45d
                               at /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/src/lib.rs:61:9
  30:     0x55fa831fd2ba - std::panicking::try::do_call::h4e5113ccc3902734
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/panicking.rs:500:40
  31:     0x55fa831fda0b - __rust_try
  32:     0x55fa831fcb9c - std::panicking::try::h56ec758ec69653d3
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/panicking.rs:464:19
  33:     0x55fa831f990c - std::panic::catch_unwind::h942098eb9228b4c4
                               at /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/panic.rs:142:14
  34:     0x55fa831f657a - LLVMFuzzerTestOneInput
                               at /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/src/lib.rs:59:22
  35:     0x55fa83207818 - _ZN6fuzzer6Fuzzer15ExecuteCallbackEPKhm
                               at /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/libfuzzer/FuzzerLoop.cpp:612:15
  36:     0x55fa83236a19 - _ZN6fuzzer10RunOneTestEPNS_6FuzzerEPKcm
                               at /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/libfuzzer/FuzzerDriver.cpp:324:21
  37:     0x55fa8323b2f4 - _ZN6fuzzer12FuzzerDriverEPiPPPcPFiPKhmE
                               at /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/libfuzzer/FuzzerDriver.cpp:860:19
  38:     0x55fa7e38d5b3 - main
                               at /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/libfuzzer/FuzzerMain.cpp:20:30
  39:     0x55fa7be23937 - __libc_start_call_main
                               at /builddir/build/BUILD/glibc-2.37/csu/../sysdeps/nptl/libc_start_call_main.h:58:16
  40:     0x55fa7be239f5 - __libc_start_main_impl
                               at /builddir/build/BUILD/glibc-2.37/csu/../csu/libc-start.c:360:3
  41:     0x55fa7e38d731 - _start
                               at /builddir/build/BUILD/glibc-2.37/csu/../sysdeps/x86_64/start.S:115
  42:                0x0 - <unknown>
==52093== ERROR: libFuzzer: deadly signal
    #0 0x55fa7e42f7f1 in __sanitizer_print_stack_trace /rustc/llvm/src/llvm-project/compiler-rt/lib/asan/asan_stack.cpp:87:3
    #1 0x55fa8321f56a in fuzzer::PrintStackTrace() /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/libfuzzer/FuzzerUtil.cpp:210:38
    #2 0x55fa83207e26 in fuzzer::Fuzzer::CrashCallback() (.part.0) /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/libfuzzer/FuzzerLoop.cpp:233:18
    #3 0x55fa83207eeb in fuzzer::Fuzzer::CrashCallback() /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/libfuzzer/FuzzerLoop.cpp:205:1
    #4 0x55fa83207eeb in fuzzer::Fuzzer::StaticCrashSignalCallback() /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/libfuzzer/FuzzerLoop.cpp:204:19
    #5 0x55fa7be3c71f  (/usr/lib64/glibc-hwcaps/x86-64-v3/libc.so.6+0x3c71f) (BuildId: d5f623d5ae0810c40b52368b29353511b554470d)
    #6 0x55fa7be9292a in __pthread_kill_implementation /builddir/build/BUILD/glibc-2.37/nptl/pthread_kill.c:43:17
    #7 0x55fa7be9292a in __pthread_kill_internal /builddir/build/BUILD/glibc-2.37/nptl/pthread_kill.c:78:10
    #8 0x55fa7be9292a in pthread_kill@@GLIBC_2.34 /builddir/build/BUILD/glibc-2.37/nptl/pthread_kill.c:89:10
    #9 0x55fa7be3c681 in gsignal /builddir/build/BUILD/glibc-2.37/signal/../sysdeps/posix/raise.c:26:13
    #10 0x55fa7be2249e in abort /builddir/build/BUILD/glibc-2.37/stdlib/abort.c:79:7
    #11 0x55fa83356f56 in std::sys::unix::abort_internal::hbd386d713b131cdf /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/sys/unix/mod.rs:359:14
    #12 0x55fa7e389566 in std::process::abort::hbfa7098f0ecb0123 /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/process.rs:2192:5
    #13 0x55fa831f80fc in libfuzzer_sys::initialize::_$u7b$$u7b$closure$u7d$$u7d$::h456beef92a984387 /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/src/lib.rs:91:9
    #14 0x55fa8334b57c in _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..Fn$LT$Args$GT$$GT$::call::h4413820818220cb9 /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/alloc/src/boxed.rs:1999:9
    #15 0x55fa8334b57c in std::panicking::rust_panic_with_hook::h90e4d15277396259 /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/panicking.rs:709:13
    #16 0x55fa8334b2d0 in std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::h2b743ea215b7c408 /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/panicking.rs:595:13
    #17 0x55fa83348805 in std::sys_common::backtrace::__rust_end_short_backtrace::h895d390908a90650 /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/sys_common/backtrace.rs:151:18
    #18 0x55fa8334b061 in rust_begin_unwind /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/panicking.rs:593:5
    #19 0x55fa7e38cc62 in core::panicking::panic_fmt::h1cc8517ae3119d0e /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/core/src/panicking.rs:67:14
    #20 0x55fa7e38ccf2 in core::panicking::panic::h8b995885e3e255d1 /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/core/src/panicking.rs:117:5
    #21 0x55fa80e8bb82 in core::cmp::Ord::clamp::ha0286db42bbb9b9b /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/core/src/cmp.rs:840:9
    #22 0x55fa80ba7a02 in rav1e::tiling::tiler::TilingInfo::from_target_tiles::hc6d98d065944c005 /home/barrbrain/rav1e/src/tiling/tiler.rs:135:26
    #23 0x55fa8073e391 in rav1e::encoder::Sequence::new::h1b3e43e4b23a5c40 /home/barrbrain/rav1e/src/encoder.rs:240:22
    #24 0x55fa7ffd44dd in rav1e::api::internal::ContextInner$LT$T$GT$::new::h133d8f040e7a27ef /home/barrbrain/rav1e/src/api/internal.rs:275:24
    #25 0x55fa7fee5e72 in rav1e::api::config::Config::new_inner::hbce573dbda0a7e6d /home/barrbrain/rav1e/src/api/config/mod.rs:241:21
    #26 0x55fa7fee806b in rav1e::api::config::Config::new_context::h801a83d55a2dd069 /home/barrbrain/rav1e/src/api/config/mod.rs:293:17
    #27 0x55fa7fa2796f in rav1e::fuzzing::fuzz_encode::h323954c405063201 /home/barrbrain/rav1e/src/fuzzing.rs:275:13
    #28 0x55fa7e463b85 in encode::_::__libfuzzer_sys_run::heebaf14ed9d99478 /home/barrbrain/rav1e/fuzz/fuzz_targets/encode.rs:18:3
    #29 0x55fa7e463496 in rust_fuzzer_test_input /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/src/lib.rs:297:60
    #30 0x55fa831f6fba in libfuzzer_sys::test_input_wrap::_$u7b$$u7b$closure$u7d$$u7d$::h1a2970ccf079a45d /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/src/lib.rs:61:9
    #31 0x55fa831fd2b9 in std::panicking::try::do_call::h4e5113ccc3902734 /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/panicking.rs:500:40
    #32 0x55fa831fda0a in __rust_try libfuzzer_sys.7e6cdf11802cca1f-cgu.4
    #33 0x55fa831fcb9b in std::panicking::try::h56ec758ec69653d3 /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/panicking.rs:464:19
    #34 0x55fa831f990b in std::panic::catch_unwind::h942098eb9228b4c4 /rustc/f0411ffcebcd7f75ac02ed45feb53ffd07b75398/library/std/src/panic.rs:142:14
    #35 0x55fa831f6579 in LLVMFuzzerTestOneInput /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/src/lib.rs:59:22
    #36 0x55fa83207817 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/libfuzzer/FuzzerLoop.cpp:612:15
    #37 0x55fa83236a18 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/libfuzzer/FuzzerDriver.cpp:324:21
    #38 0x55fa8323b2f3 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/libfuzzer/FuzzerDriver.cpp:860:19
    #39 0x55fa7e38d5b2 in main /home/barrbrain/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libfuzzer-sys-0.4.6/libfuzzer/FuzzerMain.cpp:20:30
    #40 0x55fa7be23936 in __libc_start_call_main /builddir/build/BUILD/glibc-2.37/csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #41 0x55fa7be239f4 in __libc_start_main@GLIBC_2.2.5 /builddir/build/BUILD/glibc-2.37/csu/../csu/libc-start.c:360:3
    #42 0x55fa7e38d730 in _start /builddir/build/BUILD/glibc-2.37/csu/../sysdeps/x86_64/start.S:115

NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
────────────────────────────────────────────────────────────────────────────────

Error: Fuzz target exited with exit status: 77
barrbrain commented 1 year ago

Effectively, the following assertion is failing:

assert!(min_tile_rows_ratelimit_log2 <= max_tile_rows_log2);
barrbrain commented 1 year ago

The critical detail of the configuration is that the frame rate is approximately 73,480 fps. This out-of-spec case should be identified in Config::validate() to prevent this panic.

tmatth commented 1 year ago

Reopening since #3217 was reverted.