discord / lilliput

Resize images and animated GIFs in Go
https://discord.com/blog/how-discord-resizes-150-million-images-every-day-with-go-and-c
Other
1.96k stars 124 forks source link

Nondeterministic corrupted jpeg -- mostly grey output image #31

Open huumn opened 6 years ago

huumn commented 6 years ago

When I Transform a jpeg (resize to a thumbnail) sometimes (5% or so of the time) the resulting image is a mostly grey output image ... often entirely grey. 95% of the time the same image succeeds in being resized.

There's usually an error code spit out to the logs when this happens: Corrupt JPEG data: 31283 extraneous bytes before marker 0xc4 however no error is returned from Transform

Here's the relevant code. Am I doing anything that I shouldn't be?

    inputBuf, err := ioutil.ReadAll(file)
    if err != nil {
        log.Printf("failed to read input file, %s\n", err)
        return err
    }

    decoder, err := lilliput.NewDecoder(inputBuf)
    if err != nil {
        log.Printf("error decoding image, %s\n", err)
        return err
    }
    defer decoder.Close()

    _, err = decoder.Header()
    if err != nil {
        log.Printf("error reading image header, %s\n", err)
        return err
    }

    ops := lilliput.NewImageOps(func(x, y int) int {
        if x > y {
            return x
        }
        return y
    }(image.Width, image.Height))
    defer ops.Close()

    width := int(float64(args.Height) * (float64(image.Width)/float64(image.Height)))

    opts := &lilliput.ImageOptions{
        FileType:             "." + image.Format,
        Width:                width,
        Height:               int(args.Height),
        ResizeMethod:         lilliput.ImageOpsResize,
        NormalizeOrientation: false,
        EncodeOptions:        map[string]map[int]int{
            "jpeg": map[int]int{lilliput.JpegQuality: 75},
            "png":  map[int]int{lilliput.PngCompression: 7},
            "webp": map[int]int{lilliput.WebpQuality: 75},
        }[image.Format],
    }

    // create a buffer to store the output image ... for some reason this
    // sometimes bigger than the input ...
    outputImg := make([]byte, len(inputBuf)*2)

    outputImg, err = ops.Transform(decoder, opts, outputImg)
    if err != nil {
        log.Printf("error transforming image, %s\n", err)
        return err
    }
brian-armstrong-discord commented 6 years ago

Similar to your other issue. Does your real version actually use the disk ops?

huumn commented 6 years ago

Thanks. Yes it does. No buffer reuse is occurring so I don't see the opportunity for a race. I've even run the code with -race and nothing is detected.

brian-armstrong-discord commented 5 years ago

Hi @huumn I think this might be fixed by https://github.com/discordapp/lilliput/commit/2c953373c8dc5f9b97455d7d244bd5950e67a950