Integer overflow in `stbi__jpeg_decode_block`

[JarLob]

dc * dequant[0] signed integer multiplication in stbi__jpeg_decode_block [1] overflows with a crafted image file.

Impact

It doesn't look like a potential security issue, but the signed integer overflow behavior is undefined according to C/C++ standard.

Resources

To reproduce the issue in stbi__vertical_flip_slices:

1. Make UBSAN build of the following program:

#include <stdint.h>
#define STB_IMAGE_IMPLEMENTATION
#include "../stb_image.h"

int main(int argc, char* argv[])
{
    const uint8_t data[] = {0xff,0xd8,0xff,0xdb,0x00,0x83,0x10,0xff,0xff,
                            0xc2,0x00,0x25,0x00,0x28,0x00,0x2f,0x00,0x28,
                            0x00,0x21,0x00,0x31,0x00,0x2f,0x00,0x2b,0x00,
                            0x00,0x35,0x00,0x3f,0x00,0x50,0x00,0x00,0xba,
                            0x01,0x0a,0x6b,0x01,0x0e,0x01,0x22,0x1e,0x23,
                            0x25,0x27,0x25,0x23,0x1e,0x2f,0x2f,0x33,0x33,
                            0x2f,0x2f,0x40,0x40,0x40,0x40,0x40,0x40,0x40,
                            0x40,0x40,0x40,0x40,0x40,0x40,0x40,0x01,0x11,
                            0x0f,0x0f,0x11,0x13,0x11,0x15,0x12,0x12,0x15,
                            0x14,0x11,0x14,0x11,0x14,0x1a,0x14,0x16,0x16,
                            0x14,0x1a,0x26,0x1a,0x1a,0x1c,0x1a,0x1a,0x26,
                            0x30,0x23,0x1e,0x1e,0x1e,0x1e,0x23,0x30,0x2b,
                            0x2e,0x27,0x27,0x27,0x2e,0x2b,0x35,0x35,0x30,
                            0x30,0x35,0x35,0x40,0x40,0x3f,0x40,0x40,0x40,
                            0x00,0x40,0x40,0x40,0x40,0x40,0x40,0x40,0x40,
                            0xff,0xc0,0x00,0x11,0x08,0x00,0x24,0x00,0x4c,
                            0x03,0x01,0x22,0x00,0x02,0x11,0x01,0x03,0x11,
                            0x01,0xff,0xc4,0x00,0x90,0x00,0x01,0x01,0x00,
                            0x03,0x01,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
                            0x00,0x00,0x00,0x00,0x0a,0x04,0x01,0x02,0x05,
                            0x03,0x01,0x01,0x00,0x02,0x03,0x01,0x00,0x00,
                            0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,
                            0x00,0x01,0x04,0x02,0x03,0x05,0x06,0x10,0x00,
                            0x02,0x00,0x05,0x01,0x03,0x0b,0x03,0x02,0x07,
                            0x01,0x00,0x00,0x00,0x00,0x00,0x01,0xfe,0xff,
                            0xee,0xed,0xfc,0xfb,0xde,0xd4,0x41,0x13,0x51,
                            0x71,0xd1,0x22,0x23,0xa3,0xd3,0x14,0x54,0x05,
                            0x16,0x61,0x32,0x15,0x91,0xc1,0x81,0xe1,0x42,
                            0x52,0x62,0x92,0x43,0x83,0x11,0x00,0x01,0x03,
                            0x02,0x03,0x05,0x07,0x05,0x00,0x00,0x00,0x00,
                            0x00,0x00,0x00,0x00,0x01,0x00,0x11,0x02,0xd4,
                            0x41,0x00,0x00,0x00,0x00,0x01,0x04,0x02,0x03,
                            0x05,0x97,0xd9,0x82,0xa2,0x14,0x15,0x81,0x91,
                            0x22,0x42,0x05,0xff,0xda,0x00,0x0c,0x03,0x01,
                            0x00,0x02,0x11,0x03};
    size_t size = sizeof(data);

    int x, y, channels;
    stbi_uc *img = stbi_load_from_memory(data, size, &x, &y, &channels, 4);
    stbi_image_free(img);
    return 0;
}

2. Set breakpoint at line 2225 in stbi__jpeg_decode_block and run the program to hit the overflow.

/src/stb/tests/../stb_image.h:2225:26: runtime error: signed integer overflow: -33759 * 65535 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /src/stb/tests/../stb_image.h:2225:26 in