Blur examples

[dariocravero]

Hi @jcupitt,

Thanks for making this happen! :)

I wonder if you would have any snippets of code to share to apply gaussian blur to an image. I've been looking for hours and couldn't find anything online :(.

Thanks again, Darío

[jcupitt]

Hi @dariocravero, I had a look and you're right, the thing to make a Gaussian mask is not wrapped by ruby-vips. Strange, I wonder how it got overlooked. I'm sorry you wasted so much time looking.

The next version is almost done and it's a complete rewrite of the binding. The new version is dynamic, so all vips operations are automatically wrapped. It should fix all this.

In the meantime, you need to calculate the mask yourself. I wrote a small program for you:

#!/usr/bin/ruby

include Math

require 'rubygems'
require 'vips'

# make a 1D int gaussian mask suitable for a separable convolution
#
# sigma is (roughly) radius, min_ampl is the minimum amplitude we consider, it
# sets how far out the mask goes
#
# we normalise to 20 so that the mask sum stays under 255 for most blurs ...
# this will let vips use its fast SSE path for 8-bit images

def gaussmask(sigma, min_ampl)
    sigma2 = 2.0 * sigma ** 2.0

    # find the size of the mask
    max_size = -1
    (1..10000).each do |x|
        if exp(-x ** 2.0 / sigma2) < min_ampl
            max_size = x
            break
        end
    end

    if max_size == -1
        puts "mask too large"
        return nil
    end

    width = max_size * 2 + 1
    mask = []
    (0...width).each do |x|
        xo = x - width / 2
        mask << (20.0 * exp(-xo ** 2 / sigma2)).round
    end

    puts "mask = #{mask}"

    VIPS::Mask.new [mask], mask.reduce(:+), 0
end

im = VIPS::Image.new(ARGV[0])

im = im.convsep(gaussmask(4, 0.2))

im.write(ARGV[1])

Hopefully it's clear. The 4 is roughly the radius, the 0.2 is roughly accuracy.

[jcupitt]

That function is a quick copy-paste of this C operation:

https://github.com/jcupitt/libvips/blob/master/libvips/create/gaussmat.c

You might get some more ideas from that. There are some other mask makers in that directory that might be useful.

[dariocravero]

Wow!! :) Thanks so much!! That will totally do for now... :) The difference with MiniMagick is huge. Here's the test program:

#!/usr/bin/ruby

require 'mini_magick'

image = MiniMagick::Image.open ARGV[0]
image.gaussian_blur 60
image.write ARGV[1]

Here are the time outputs while blurring an image with a radius of 60:

ruby-vips:

real    0m9.505s
user    0m35.998s
sys 0m0.124s

mini_magick:

real    5m54.084s
user    5m53.288s
sys 0m0.474s

That's on a MacBook Pro with a 2.3 GHz i5 and 16GB of memory. ruby-vips is nearly 12 times faster! Amazing! :dancer:

Dynamic bindings are probably the best, @DAddYE never released ffi-gen but it's supposed to do just that :). Do you need any help with that or the docs or something else? Let me know. Glad to help :)

[jcupitt]

If you want to work on the new ruby binding, that would be great!

I've almost done the Python one, it would make a good starting point:

https://github.com/jcupitt/libvips/blob/master/python/vips8/vips.py

It's based on gobject-introspection:

https://developer.gnome.org/gi/stable/

The idea is that the C library is marked up with some special comments. These are parsed by gobject-introspection to generate a typelib. The API docs are generated from these comments too, eg.:

http://www.vips.ecs.soton.ac.uk/supported/7.40/doc/html/libvips/libvips-conversion.html#vips-join

The typelib is loaded by a Python program using pygobject:

https://wiki.gnome.org/action/show/Projects/PyGObject

Or a Ruby program using the equivalent gem:

https://rubygems.org/gems/gobject-introspection

Now you can call directly into the C library from Ruby. The API ends up being rather un-Pythonic (or rather un-Ruby-ish), so you write a small layer over that to make a nice, high-level binding. vips8 has some extra introspection stuff it provides to expose features like optional arguments and default values.

You end up with this nice API, all generated at runtime in only a few hundred lines of Python, with no C required. This should make the binding more portable, hopefully. Platforms like Windows will finally get support.

a = Vips.Image.new_from_file(sys.argv[1])

b = Vips.Image.new_from_file(sys.argv[2])

c = a.join(b, Vips.Direction.HORIZONTAL, 
           expand = True, 
           shim = 100, 
           align = Vips.Align.CENTRE, 
           background = [128, 255, 128])

c.write_to_file(sys.argv[3])

There are some test Python files which show the process. try.py just uses plain gobject-introspection, plus vips introspection to implement a function which can call any vips operation:

https://github.com/jcupitt/libvips/blob/master/python/try.py

Then vips8.py uses more-or-less that, but overrides getattr on the Image class so that a.thing(b) ends up as Vips.vips_call("thing", a, b).

My current work plan is:

• Write a vips8 test suite in Python. The vips7 one was done in nip2 and had some problems. I'm about 1/3 of the way through this.
• Make a C++ binding (just add smart pointers to the C one, I think, it won't be much work).
• Make a Ruby binding along the lines of the Python one.
• Declare vips8 finished!

I'd be very happy to hand you the Ruby part, if you have time.

[jcupitt]

Also, try your benchmark on a large image, perhaps 10,000 x 10,000 pixel RGB jpeg. You'll see a huge difference in memory use as well.

If you add :sequential => true to your Image.new it'll turn on sequential mode and you should see a further drop in memory use and a bit more speed. There was a blog post about sequential mode, if you've not seen it:

http://libvips.blogspot.co.uk/2012/06/how-libvips-opens-file.html

[jcupitt]

Last post, I'm not sure I was very clear about the gobject-introspection stuff.

vips8.py uses goi to call the 'core' vips8 API. This part of vips8 should be pretty unchanging as the library evolves in the future.

It uses the vips8 introspection stuff, invoked via goi, to look for and call vips8 operations, like vips_join() (join two images together). The set of operations will change: they will gain new optional arguments, new operations will be added (they can even be added at runtime via plugins), so this part of the binding is extremely dynamic.

Summary: the vips8 Python and Ruby bindings should automatically update as needed in the future. They are written in pure Python (or Ruby) so should be trivially portable. They should only be a few hundred lines of code.

It should be possible to generate the docs automatically too, but I've not really looked into that yet.

[dariocravero]

Brilliant! Thanks for the detailed explanation on how to get that going. Introspecting the library to automatically build the bindings is very clever. I will try to give it a go any time soon but can't promise anything as we're currently in the middle of releasing a good few things over the next few weeks. What's your expected timeline on it? I could probably schedule it in :)

[dariocravero]

Hey John,

Here's a more Ruby-esque version of the mask:

NORMALISE_TO = 20.0
BIGGEST_MASK = 10000
def gaussmask2(sigma, min_ampl)
  sigma2 = 2.0 * sigma ** 2.0

  # find the size of the mask
  max_size = (1..BIGGEST_MASK).detect { |x| Math::exp(-x ** 2.0 / sigma2) < min_ampl }
  throw :mask_too_large unless max_size

  width = max_size * 2 + 1
  mask = (0...width).map do |x|
    d = (x - width / 2) ** 2
    (NORMALISE_TO * Math::exp(-d / sigma2)).round
  end
  sum = mask.reduce(:+)

  VIPS::Mask.new [mask], sum, 0
end

Any thoughts on it? Hope you like it :).

The constants would generally be extracted into some sort of class but that will do for the example. Taking carrierwave-vips as a base, I'm making it agnostic of the uploader (or file manager) and building an operation-oriented processing layer. Should be releasing it today :).

EDIT: replaced sum = mask.reduce(&:+) for sum = mask.reduce(:+) as we don't need the &.

[jcupitt]

Oh much neater, nice. Why do you need the & before the :+? I like Ruby, but I'm not much good at it :(

I'll be starting the ruby-vips8 binding in a couple of months, so make a start before then if you'd like to take it over.

[dariocravero]

As a matter of fact, you don't. I guess auld habits die hard :P :). reduce doesn't need it.

Good, will take that into account then!

[dariocravero]

Gem released! vips-process. GitHub repo. Would love to hear your thoughts on it @jcupitt :)

[jcupitt]

Wow nice! That's much more Ruby-esque than anything I've tried.

I noticed one tiny thing on a quick read, you have:

# @param sigma Integer roughly the radius

Of course sigma (the standard deviation of the gaussian) is a float. Don't suppose it makes much difference.

[dariocravero]

Cool :) Updated!.

[jcupitt]

I read a bit more. The README is getting easier to understand, heh. It's a nice way to specify a set of operations, it feels very declarative.

• vips8 has a fancy operation cache: it memoises the last 1,000 operators and, if you repeat one, it'll give you the old one back rather than running again. You could save yourself writing the version cache system if ruby-vips8 happens.
• vips8 has read and write, to and from memory buffers. It'd be nice to be able to support this, it's handy for cloud stuff where the filesystem is often relatively more expensive. See:

http://www.vips.ecs.soton.ac.uk/supported/7.40/doc/html/libvips/VipsImage.html#vips-image-new-from-buffer

• What about versions that return non-image objects, like find-image-maximum? I suppose you keep that kind of thing inside versions.
• You always turn on sequential mode, so you can't do things like flip up-down. Perhaps you could attach a note to an image which says if it's in sequential mode, then before a non-sequential operation, like flip up-down, transform the sequential image to a random access one.
• You don't use JPEG shrink-on-load, so thumbnailing will be rather slow. You'd need to have some way for something like resize to reopen its input at lower resolution.

[jcupitt]

I did a blog post about the new Python binding:

http://libvips.blogspot.co.uk/2014/10/image-annotation-with-pyvips8.html

It has some timings and examples. The Ruby vips8 binding should get nicer in a similar way, hopefully.

[dariocravero]

Sorry for the late reply. Thanks for the feedback. I reckon that ruby-vips8 will be a great addon. I'm looking forward to having some time to come around but it's unlikely for the following months. From what I can see on the post about Python, vips8 looks very exciting :).

Regarding the suggestions at the end of your previous post:

• What about versions that return non-image objects, like find-image-maximum? I suppose you keep that kind of thing inside versions.

I haven't thought of those as I haven't had a use case but probably the answer will be yes. I think I have one coming up soon though: get the predominant colour on an image. We'll see what comes out next.

• You always turn on sequential mode, so you can't do things like flip up-down. Perhaps you could attach a note to an image which says if it's in sequential mode, then before a non-sequential operation, like flip up-down, transform the sequential image to a random access one.

True, need to have a look at that.

• You don't use JPEG shrink-on-load, so thumbnailing will be rather slow. You'd need to have some way for something like resize to reopen its input at lower resolution.

How would you go about implementing that? Also, do you have any recommendations on smarter resampling methods (downsizing is sometimes crippling the image a bit). Thanks again! :)

[jcupitt]

vipsthumbnail uses jpeg-shrink-on-load. It opens once to get the true image dimensions, calculates the shrink factor, then opens again, setting "shrink".

Resampling methods: vipsthumbnail has a better one now, check the sources. The idea is to .shrink() less and .affine() more. This tends to preserve edges better. To prevent aliasing, you put a slight blur inbetween them. The lower sampling density gives a peak, the blur makes some lobes, and you end up with something close to lanczos2, the default ImageMagick shrinker. It's noticably better quality than the previous technique I was using.

[dariocravero]

Perfect will make sure to check that out. Thanks!

[jcupitt]

I came across this old issue by accident. ruby-vips8 is finally out as a gem:

http://libvips.blogspot.co.uk/2016/01/ruby-vips-is-dead-long-live-ruby-vips8.html