Issues with JPEG2000 every second picture turns images black and white

[kaspergrubbe]

Describe the bug When I run the following the code, every second picture becomes black and white.

To Reproduce

Here's the .jp2 file: ernie-the-staffie.jp2.zip

Run the following code:

ENV['VIPS_CONCURRENCY'] = '1'
ENV['VIPS_NOVECTOR'] = '1'
ENV['VIPS_INFO'] = '1'

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'ruby-vips', '2.1.4'
end

require 'vips'

def vips_to_file(width)
  filename = "ernie-#{Time.now.to_i}-#{width}.jp2"
  puts "Filename: #{filename}"
  vips_image = Vips::Image.new_from_file('ernie-the-staffie.jp2')
  # Height is set stupidly high because of https://github.com/jcupitt/ruby-vips/issues/150
  vips_image = vips_image.thumbnail_image(width, height: 10000000)
  vips_image.write_to_file(filename)
  puts ''
end

vips_to_file(500)
vips_to_file(999)
vips_to_file(1000)
vips_to_file(1001)
vips_to_file(1002)
vips_to_file(1003)

Output:

VIPS-INFO: 00:20:29.009: g_getenv( "PATH" ) == "/Users/kg/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/bin:/Users/kg/.rbenv/versions/3.1.1/bin:/Users/kg/.rbenv/libexec:/Users/kg/.rbenv/plugins/ruby-build/bin:/usr/local/opt/node@12/bin:/usr/local/opt/node@14/bin:/Users/kg/.rbenv/shims:/Users/kg/.rbenv/bin:/usr/local/opt/ncurses/bin:/usr/local/opt/qt@5.5/bin:/usr/local/share/npm/bin:/Users/kg/.bin:/usr/local/bin:/usr/local:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/sbin:/Users/kg/.go_packages/bin:/usr/local/opt/go/libexec/bin"
VIPS-INFO: 00:20:29.009: looking in "/Users/kg/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/Users/kg/.rbenv/versions/3.1.1/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/Users/kg/.rbenv/libexec" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/Users/kg/.rbenv/plugins/ruby-build/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/usr/local/opt/node@12/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/usr/local/opt/node@14/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/Users/kg/.rbenv/shims" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/Users/kg/.rbenv/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/usr/local/opt/ncurses/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/usr/local/opt/qt@5.5/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/usr/local/share/npm/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/Users/kg/.bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/usr/local/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/usr/local" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/usr/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/usr/sbin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/sbin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/opt/X11/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/usr/local/sbin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/Users/kg/.go_packages/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: looking in "/usr/local/opt/go/libexec/bin" for "vips.rb"
VIPS-INFO: 00:20:29.009: trying for dir = "/Users/kg/projects/vipserror/vips.rb", name = "vips.rb"
VIPS-INFO: 00:20:29.009: canonicalised path = "/Users/kg/projects/vipserror"
VIPS-INFO: 00:20:29.009: VIPS_PREFIX = /usr/local/Cellar/vips/8.12.2_2
VIPS-INFO: 00:20:29.009: VIPS_LIBDIR = /usr/local/Cellar/vips/8.12.2_2/lib
VIPS-INFO: 00:20:29.009: prefix = /usr/local/Cellar/vips/8.12.2_2
VIPS-INFO: 00:20:29.009: libdir = /usr/local/Cellar/vips/8.12.2_2/lib
VIPS-INFO: 00:20:29.010: searching "/usr/local/Cellar/vips/8.12.2_2/lib/vips-plugins-8.12"

Filename: ernie-1658445629-500.jp2
VIPS-INFO: 00:20:29.029: selected loader is image source
VIPS-INFO: 00:20:29.029: input size is 500 x 500
VIPS-INFO: 00:20:29.029: loading with factor 1 pre-shrink
VIPS-INFO: 00:20:29.029: pre-shrunk size is 500 x 500
VIPS-INFO: 00:20:29.029: converting to processing space srgb

Filename: ernie-1658445629-999.jp2
VIPS-INFO: 00:20:29.158: selected loader is image source
VIPS-INFO: 00:20:29.158: input size is 500 x 500
VIPS-INFO: 00:20:29.158: loading with factor 1 pre-shrink
VIPS-INFO: 00:20:29.158: pre-shrunk size is 500 x 500
VIPS-INFO: 00:20:29.158: converting to processing space srgb
VIPS-INFO: 00:20:29.158: residual scale 1.998 x 1.998

Filename: ernie-1658445629-1000.jp2
VIPS-INFO: 00:20:29.402: selected loader is image source
VIPS-INFO: 00:20:29.402: input size is 500 x 500
VIPS-INFO: 00:20:29.402: loading with factor 1 pre-shrink
VIPS-INFO: 00:20:29.402: pre-shrunk size is 500 x 500
VIPS-INFO: 00:20:29.402: converting to processing space srgb
VIPS-INFO: 00:20:29.402: residual scale 2 x 2

Filename: ernie-1658445629-1001.jp2
VIPS-INFO: 00:20:29.634: selected loader is image source
VIPS-INFO: 00:20:29.634: input size is 500 x 500
VIPS-INFO: 00:20:29.634: loading with factor 1 pre-shrink
VIPS-INFO: 00:20:29.634: pre-shrunk size is 500 x 500
VIPS-INFO: 00:20:29.634: converting to processing space srgb
VIPS-INFO: 00:20:29.634: residual scale 2.002 x 2.002

Filename: ernie-1658445629-1002.jp2
VIPS-INFO: 00:20:29.870: selected loader is image source
VIPS-INFO: 00:20:29.870: input size is 500 x 500
VIPS-INFO: 00:20:29.870: loading with factor 1 pre-shrink
VIPS-INFO: 00:20:29.870: pre-shrunk size is 500 x 500
VIPS-INFO: 00:20:29.870: converting to processing space srgb
VIPS-INFO: 00:20:29.870: residual scale 2.004 x 2.004

Filename: ernie-1658445630-1003.jp2
VIPS-INFO: 00:20:30.091: selected loader is image source
VIPS-INFO: 00:20:30.091: input size is 500 x 500
VIPS-INFO: 00:20:30.091: loading with factor 1 pre-shrink
VIPS-INFO: 00:20:30.091: pre-shrunk size is 500 x 500
VIPS-INFO: 00:20:30.091: converting to processing space srgb
VIPS-INFO: 00:20:30.092: residual scale 2.006 x 2.006

Expected behavior I expect the colour to stay on scaled images.

Screenshots

I can't attach the actual images since Github doesn't support JPEG2000 (I've put them in the zip): ernie-the-staffie.zip and here's a screenshot:

[kaspergrubbe]

I can replicate it with vipsthumbnail so it's not only limited to ruby-vips:

$ vipsthumbnail -v
vips-8.12.2-Tue Jan 25 09:34:32 UTC 2022

$ vipsthumbnail ernie-the-staffie.jp2 --vips-info --size 500x -o 500.jp2
VIPS-INFO: 01:06:59.199: thumbnailing ernie-the-staffie.jp2
VIPS-INFO: 01:06:59.201: selected loader is VipsForeignLoadJp2kFile
VIPS-INFO: 01:06:59.201: input size is 500 x 500
VIPS-INFO: 01:06:59.207: loading with factor 0 pre-shrink
VIPS-INFO: 01:06:59.207: pre-shrunk size is 500 x 500
VIPS-INFO: 01:06:59.207: converting to processing space srgb
VIPS-INFO: 01:06:59.207: thumbnailing ernie-the-staffie.jp2 as ./500.jp2

$ vips --vips-config
enable debug: no
enable deprecated library components: yes
enable modules: no
use fftw3 for FFT: yes
accelerate loops with orc: yes
ICC profile support with lcms: yes (lcms2)
zlib: yes
text rendering with pangocairo: yes
font file support with fontconfig: yes
RAD load/save: yes
Analyze7 load/save: yes
PPM load/save: yes
GIF load:  yes
GIF save with cgif: yes
EXIF metadata support with libexif: yes
JPEG load/save with libjpeg: yes (pkg-config)
JXL load/save with libjxl: yes (dynamic module: no)
JPEG2000 load/save with libopenjp2: yes
PNG load with libspng: yes
PNG load/save with libpng: yes (pkg-config libpng >= 1.2.9)
quantisation to 8 bit: yes
TIFF load/save with libtiff: yes (pkg-config libtiff-4)
image pyramid save: yes
HEIC/AVIF load/save with libheif: yes (dynamic module: no)
WebP load/save with libwebp: yes
PDF load with PDFium:  no
PDF load with poppler-glib: yes (dynamic module: no)
SVG load with librsvg-2.0: yes
EXR load with OpenEXR: yes
OpenSlide load: yes (dynamic module: no)
Matlab load with matio: yes
NIfTI load/save with niftiio: no
FITS load/save with cfitsio: yes
Magick package: MagickCore (dynamic module: no)
Magick API version: magick7
load with libMagickCore: yes
save with libMagickCore: yes

[jcupitt]

Hi @kaspergrubbe,

That's a weird one. It's working for me with libvips 8.13 on ubuntu 22.04, I'll try a few other versions.

What versions of openjpeg and libvips are you using, and what program are you using to view your jp2 files?

[jcupitt]

Yes, it's working with libvips 8.12 as well. This is with openjpeg 2.4.0-6 (the one ubuntu ship with 22.04).

[kaspergrubbe]

I am viewing the files with my browser, and the in-built filebrowser and previewer of OSX.

Vips version:

$ vips -v
vips-8.12.2-Tue Jan 25 09:34:32 UTC 2022

This is the info about openjpeg:

$ brew info --json openjpeg
    "installed": [
      {
        "version": "2.4.0",
        "used_options": [

        ],
        "built_as_bottle": true,
        "poured_from_bottle": true,
        "runtime_dependencies": [
          {
            "full_name": "libpng",
            "version": "1.6.37"
          },
          {
            "full_name": "jpeg",
            "version": "9d"
          },
          {
            "full_name": "libtiff",
            "version": "4.2.0"
          },
          {
            "full_name": "little-cms2",
            "version": "2.12"
          }
        ],
        "installed_as_dependency": true,
        "installed_on_request": false
      }
    ],

I'll ty and update my dependencies.

[kaspergrubbe]

I've updated openjpeg to version 2.5.0.

I've figured out that it isn't every second picture, it seems to be correlated with the width of the outputted images:

Unrelated, but is it normal that the residiual scale is only there for some cases with the same input/output?

Filename: ernie-1658484258-1003.jp2
VIPS-INFO: 11:04:18.047: selected loader is image source
VIPS-INFO: 11:04:18.047: input size is 500 x 500
VIPS-INFO: 11:04:18.047: loading with factor 1 pre-shrink
VIPS-INFO: 11:04:18.047: pre-shrunk size is 500 x 500
VIPS-INFO: 11:04:18.047: converting to processing space srgb
**VIPS-INFO: 11:04:18.047: residual scale 2.006 x 2.006**

Filename: ernie-1658484259-1003.jp2
VIPS-INFO: 11:04:19.285: selected loader is image source
VIPS-INFO: 11:04:19.285: input size is 500 x 500
VIPS-INFO: 11:04:19.285: loading with factor 1 pre-shrink
VIPS-INFO: 11:04:19.285: pre-shrunk size is 500 x 500
VIPS-INFO: 11:04:19.285: converting to processing space srgb

[kaspergrubbe]

I've tried to update and rebuild all libvips dependencies:

$ brew deps libvips | xargs brew remove --ignore-dependencies
$ brew remove libvips
$ brew install libvips

But vipsthumbnail still produces a grey image:

$ vipsthumbnail ernie-the-staffie.jp2 --vips-info --size 500x -o 500.jp2
VIPS-INFO: 11:25:59.612: thumbnailing ernie-the-staffie.jp2
VIPS-INFO: 11:25:59.614: selected loader is VipsForeignLoadJp2kFile
VIPS-INFO: 11:25:59.614: input size is 500 x 500
VIPS-INFO: 11:25:59.620: loading with factor 0 pre-shrink
VIPS-INFO: 11:25:59.620: pre-shrunk size is 500 x 500
VIPS-INFO: 11:25:59.620: converting to processing space srgb
VIPS-INFO: 11:25:59.620: thumbnailing ernie-the-staffie.jp2 as ./500.jp2

Do you have any other suggestions for me?

[kaspergrubbe]

My hypothesis at the moment: If the image width is divisible by 2, then it produces a grey image 😅

[jcupitt]

I think it might be your viewer program failing to perform chroma upsampling correctly for even image sizes. Both Safari (I guess that's your browser) and the macOS preview app will be using the same jpeg2000 decoder.

Could you try reading the grey images with vips? eg. for me, this:

$ vips copy ernie-1658462479-500.jp2 x.png

Makes:

[jcupitt]

Unrelated, but is it normal that the residiual scale is only there for some cases with the same input/output?

Huh, I can't reproduce this. What command did you run?

[kaspergrubbe]

Huh, I can't reproduce this. What command did you run?

ENV['VIPS_CONCURRENCY'] = '1'
ENV['VIPS_NOVECTOR'] = '1'
ENV['VIPS_INFO'] = '1'

require 'bundler/inline'
gemfile do
  source 'https://rubygems.org'
  gem 'ruby-vips', '2.1.4'
end

require 'vips'

def vips_to_file(width)
  filename = "ernie-#{Time.now.to_i}-#{width}.jp2"
  puts "Filename: #{filename}"
  vips_image = Vips::Image.new_from_file('ernie-the-staffie.jp2')
  # Height is set stupidly high because of https://github.com/jcupitt/ruby-vips/issues/150
  vips_image = vips_image.thumbnail_image(width, height: 10_000_000)
  vips_image.write_to_file(filename)
  puts ''
  sleep 1
end

vips_to_file(1003)
vips_to_file(1003)
vips_to_file(1003)

Produces this (the first run uses residiual scale):

VIPS-INFO: 12:00:16.787: g_getenv( "PATH" ) == "/Users/kg/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/bin:/Users/kg/.rbenv/versions/3.1.1/bin:/Users/kg/.rbenv/libexec:/Users/kg/.rbenv/plugins/ruby-build/bin:/usr/local/opt/node@12/bin:/usr/local/opt/node@14/bin:/Users/kg/.rbenv/shims:/Users/kg/.rbenv/bin:/usr/local/opt/ncurses/bin:/usr/local/opt/qt@5.5/bin:/usr/local/share/npm/bin:/Users/kg/.bin:/usr/local/bin:/usr/local:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/sbin:/Users/kg/.go_packages/bin:/usr/local/opt/go/libexec/bin"
VIPS-INFO: 12:00:16.787: looking in "/Users/kg/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/Users/kg/.rbenv/versions/3.1.1/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/Users/kg/.rbenv/libexec" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/Users/kg/.rbenv/plugins/ruby-build/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/usr/local/opt/node@12/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/usr/local/opt/node@14/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/Users/kg/.rbenv/shims" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/Users/kg/.rbenv/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/usr/local/opt/ncurses/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/usr/local/opt/qt@5.5/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/usr/local/share/npm/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/Users/kg/.bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/usr/local/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/usr/local" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/usr/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/usr/sbin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/sbin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/opt/X11/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/usr/local/sbin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/Users/kg/.go_packages/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: looking in "/usr/local/opt/go/libexec/bin" for "vips.rb"
VIPS-INFO: 12:00:16.787: trying for dir = "/Users/kg/projects/vipserror/vips.rb", name = "vips.rb"
VIPS-INFO: 12:00:16.787: canonicalised path = "/Users/kg/projects/vipserror"
VIPS-INFO: 12:00:16.791: VIPS_PREFIX = /usr/local/Cellar/vips/8.12.2_2
VIPS-INFO: 12:00:16.791: VIPS_LIBDIR = /usr/local/Cellar/vips/8.12.2_2/lib
VIPS-INFO: 12:00:16.791: prefix = /usr/local/Cellar/vips/8.12.2_2
VIPS-INFO: 12:00:16.791: libdir = /usr/local/Cellar/vips/8.12.2_2/lib
VIPS-INFO: 12:00:16.792: searching "/usr/local/Cellar/vips/8.12.2_2/lib/vips-plugins-8.12"

Filename: ernie-1658487616-1003.jp2
VIPS-INFO: 12:00:16.824: selected loader is image source
VIPS-INFO: 12:00:16.824: input size is 500 x 500
VIPS-INFO: 12:00:16.824: loading with factor 1 pre-shrink
VIPS-INFO: 12:00:16.824: pre-shrunk size is 500 x 500
VIPS-INFO: 12:00:16.824: converting to processing space srgb
VIPS-INFO: 12:00:16.824: residual scale 2.006 x 2.006

Filename: ernie-1658487618-1003.jp2
VIPS-INFO: 12:00:18.074: selected loader is image source
VIPS-INFO: 12:00:18.074: input size is 500 x 500
VIPS-INFO: 12:00:18.075: loading with factor 1 pre-shrink
VIPS-INFO: 12:00:18.075: pre-shrunk size is 500 x 500
VIPS-INFO: 12:00:18.075: converting to processing space srgb

Filename: ernie-1658487619-1003.jp2
VIPS-INFO: 12:00:19.315: selected loader is image source
VIPS-INFO: 12:00:19.315: input size is 500 x 500
VIPS-INFO: 12:00:19.315: loading with factor 1 pre-shrink
VIPS-INFO: 12:00:19.315: pre-shrunk size is 500 x 500
VIPS-INFO: 12:00:19.315: converting to processing space srgb

[kaspergrubbe]

I think it might be your viewer program failing to perform chroma upsampling correctly for even image sizes. Both Safari (I guess that's your browser) and the macOS preview app will be using the same jpeg2000 decoder.

Could you try reading the grey images with vips? eg. for me, this:

$ vips copy ernie-1658462479-500.jp2 x.png

I did this, and the resulting png have colours again!

You're right! It must be a bug somewhere in OSX! Firefox, my file browser, Safari and my API-test tool Paw must all use the same library and thus they all show the image as missing colours!

[jcupitt]

Produces this (the first run uses residiual scale):

Ah, that's the libvips operation cache. It'll reuse the previous result if you run again with the same arguments.

Please don't use thumbnail_image: it's only there for resizing computed images. The vanilla thumbnail operation combines load and resize in one step, so it can exploit tricks like shrink-on-load. It's also able to overlap decode and recode, so memory use will be quite a bit lower (and speed doubled).

I'd write:

def vips_to_file(width)
  filename = "ernie-#{Time.now.to_i}-#{width}.jp2"
  puts "Filename: #{filename}"
  # Height is set stupidly high because of https://github.com/jcupitt/ruby-vips/issues/150
  vips_image = Vips::Image.thumbnail('ernie-the-staffie.jp2', width, height: 10000000)
  vips_image.write_to_file(filename)
  puts ''
end

[jcupitt]

I did this, and the resulting png have colours again!

Hooray!

So as a workaround I guess you need to pad images to an odd width before saving them. You can pad with eg.:

def vips_to_file(width)
  filename = "ernie-#{Time.now.to_i}-#{width}.jp2"
  puts "Filename: #{filename}"
  # Height is set stupidly high because of https://github.com/jcupitt/ruby-vips/issues/150
  vips_image = Vips::Image.thumbnail('ernie-the-staffie.jp2', width, height: 10000000)
  vips_image.
    gravity(:north_west,
            vips_image.width + 1 - vips_image.width % 2,
            vips_image.height,
            extend: :copy).
    write_to_file(filename)
  puts ''
end

What a strange thing.

[kaspergrubbe]

What an adventure! :) Thanks for debugging with me, have a wonderful weekend!