libvips / lua-vips

Lua binding for the libvips image processing library
MIT License
125 stars 10 forks source link

Unable to change pixel interpretation #76

Closed Tychology closed 5 months ago

Tychology commented 5 months ago

I want to calculate the magnitude and angle of the gradient of an image and display the magnitude as the value and the angle as the hue of HSV. But because set("interpretation", "multiband") and set("interpretation", "hsv") doesn't seem to do anything, when I load an srgb image I get an image back that is still incorrectly interpreted as srgb by write_to_file

local vips = require("vips")

local xkernel = vips.Image.new_from_array { { -3, 0, 3 }, { -10, 0, 10 }, { -3, 0, 3 } }
local ykernel = xkernel:rot90()

-- input 3band srgb image
function grad(image)
    image:set("interpretation", "multiband")
    -- image still srgb. why?

    image = image:bandmean()
    -- imgage 1band srgb

    local xgrad = image:conv(xkernel)
    local ygrad = image:conv(ykernel)

    local magnitude = (xgrad ^ 2 + ygrad ^ 2) ^ 0.5
    local angle = ygrad:math2(xgrad, "atan2")
    -- xgrad, ygrad, magnitude and angle are 1band srgb

    return magnitude .. angle
    --returns input 2 band srgb image instead of 2 band multiband what I want
end

-- input 2 band srgb instead of 2 band multiband
function ma2HSV(ma)
    local hsv = ma:extract_band(1)
    hsv = hsv .. 256
    hsv = hsv .. ma:extract_band(0)
    hsv:set("interpretation", "hsv")
    -- hsv is still 3 band srgb instead of hsv
    return hsv
end

local image = vips.Image.new_from_file("image.png")
-- image is 3 band srgb like it should be

ma2HSV(grad(image)):write_to_file("grad.png")
-- because of the incorrect interpretation, this is treated as an srgb image, not a hsv image

I really don't understand why set("interpretation", does't work. Trying to set using an invalid enum value gives this error message which indicates that this is the correct way of setting the interpretation attribute of an image object: lua: /usr/local/share/lua/5.4/vips/gvalue.lua:76: no such enum invalid lua-vips: enum 'VipsInterpretation' has no member 'invalid', should be one of: error, multiband, b-w, histogram, xyz, lab, cmyk, labq, rgb, cmc, lch, labs, srgb, yxy, fourier, rgb16, grey16, matrix, scrgb, hsv

Here is my input and output: image grad

RiskoZoSlovenska commented 5 months ago

Hello! Image:set only sets image metadata, i.e. what colourspace the image is in right now. To convert an image to a different interpretation, you'll want to look at Image:colourspace.

Tychology commented 5 months ago

I know, I only want to set the image metadata, not invoke a colourspace conversion, just treat the data differently. Since I basically want to assemble an hsv from parts and tell vips to actually treat it it as hsv. the problem is that calling Image:set doesn't set the interpretation metadata for some reason. I checked with a debugger and Image:get. the interpretation field is just not changed after Image:set.

jcupitt commented 5 months ago

Hiya, copy() is the best thing to use for this, eg.:

img = img:copy{interpretation = "srgb"}

Behind the scenes it's just a pointer copy, so this is very fast.

libvips is (almost) all immutable, so you make new objects, you don't modify existing objects.

You can use set(), but you have to be very careful, since pointers can be shared through the system by the various caches and you might be modifying an image some other part of your program is using. You need to make sure you have a private and unique image first with copy().

I think set() also won't set the builtin header fields like width and interpretation, since it's so easy to trigger a hard crash. copy() lets you set these things, but applies a set of sanity checks which prevent you changing the size of pixels, or addressing out of range memory.

jcupitt commented 5 months ago

The ruby binding has a feature for this:

https://www.libvips.org/2021/03/08/ruby-vips-mutate.html

The mutate block lets you create an image which can be modified safely. It'd be neat if lua-vips could have something similar, though I suppose the syntax wouldn't be so nice.

Tychology commented 5 months ago

Thank you very much, it worked! grad If set() doesn't work on certain field I think It would be a good idea to have that documented or at least not fail silently but with a warning or even an error.

jcupitt commented 5 months ago

Haha, your parrots look great!

I tried to improve the docs for set and remove.

jcupitt commented 5 months ago

This is off-topic, but LCh is a better colourspace than HSV, it might be worth trying that. It's perceptually uniform, so hue changes ought to match angle changes in a more obvious way.

https://en.wikipedia.org/wiki/HCL_color_space

rolandlo commented 5 months ago

I'm closing the issue, since the docs update has been merged into master.