Usage with OpenResty?

[tomas]

Congrats on your work, lua-vips looks really nice. I'd like to use it with OpenResty, but I want to avoid blocking the main process. Have you done any tests in this area and/or have any ideas on how you would approach this?

[jcupitt]

Hello, sorry, I don't know anything about OpenResty.

Is it much like node? node-vips is asynchronous, so it should be possible:

https://github.com/jcupitt/node-vips

All you do is start a thread to do the computation (computation will always happen from call(), so you only need to add this in one place) and run a OpenResty callback (I guess?) when the operation returns.

[tomas]

I'm not sure if that's possible, but I'll see what I can do. In this thread, some people suggest using a background queue and/or possibly running it in another process via sockproc, but I was looking for a simpler solution.

Anyway, I'll do a quick test and see how it goes. I just wanted to check if this was something you had on your radar or not.

And thanks for the quick answer, by the way!

[jcupitt]

Would this work?

https://github.com/torch/threads

It looks like you can fire off background threads and get a callback on the main thread when the background task completes.

[tomas]

It might! I'll give it a shot and let you know if it works.

[tomas]

Quick question: any idea why I might be seeing this?

(.:9335): GLib-GObject-WARNING **: cannot register existing type 'VipsObject'

(.:9335): GLib-CRITICAL **: g_once_init_leave: assertion 'result != 0' failed

(.:9335): GLib-GObject-CRITICAL **: g_type_register_static: assertion 'parent_type > 0' failed

(.:9335): GLib-CRITICAL **: g_once_init_leave: assertion 'result != 0' failed

The process seems to hang after that, and doesn't respond to any more requests.

[jcupitt]

I'd guess it's being inited in parallel, or perhaps not shut down properly, then inited again.

What are you doing to cause the error?

[jcupitt]

Thinking a bit more, I'd guess you are running require "vips" in a worker. You need to init vips just once from the main thread, then pass the vips handle down to each worker.

[dmitrytretyakov]

@tomas How did you use lua-vips with openresty? Did you have a problem with memory leak?

[maxim-avramenko]

Quick question: any idea why I might be seeing this?

(.:9335): GLib-GObject-WARNING **: cannot register existing type 'VipsObject'

(.:9335): GLib-CRITICAL **: g_once_init_leave: assertion 'result != 0' failed

(.:9335): GLib-GObject-CRITICAL **: g_type_register_static: assertion 'parent_type > 0' failed

(.:9335): GLib-CRITICAL **: g_once_init_leave: assertion 'result != 0' failed

The process seems to hang after that, and doesn't respond to any more requests.

You see this error message because you connect the "vips" inside each worker_process what nginx start, Most likely you wright require("vips") inside the worker_process, for example like this: local vips = require("vips") inside file.lua in content_by_lua_file directive. This error thrown because "vips" trying to register existing (already registered in first location request) type 'VipsObject'. We can translate Error message like "I already have the same, enough for me, please stop,I beg you, stop! dont do this again!" or something like that.

To fix this error you must add init_by_lua_block before server {} directive in your nginx.conf and add there require "vips" command:

# use this directive to init your luarocks, all other require local vips = require "vips" inside all worker_processes will use this preloaded luarocks, i think this is similar singleton objects in OOP to prevent create more than one object in application
init_by_lua_block {
    require "cjson";
    require "vips";
    require "magick";
    -- some other luarocks here
}
# now every new worker_process will use one luarock that was init only one time when nginx start
server {
    listen       80;
    lua_code_cache on;
    server_name  localhost;
    ..... 
    other directives
    .....
}

When you add init_by_lua_block, and add your luarocks list in it, your local vips = require "vips" inside file.lua in content_by_lua_file directive will use "vips" from init_by_lua_block.

Use local vips = require "vips" instead local vips = require("vips")

You can read more about this behavior in documentation: https://github.com/openresty/lua-nginx-module#init_by_lua

[jcupitt]

Ah, very interesting. Thank you @maxim-avramenko !

[maxim-avramenko]

Ah, very interesting. Thank you @maxim-avramenko !

U R welcome ;) i use luarock "magick" to thumbnails my images for web, but i dont know how fast "magick" and decided to find another tool to make thumbs and test which tool faster. lua-vips vs magick, each of them has advantages and disadvantages. "lua-vips" has beter functionality but not easy to deploy with Docker ;) "magick" has less functionality but deploy with Docker very simple

i think it is good idea to update lua-vips wiki with instruction "how to deploy with Docker", it will save millions of hours to a lot of people

[jcupitt]

I've not done much with openresty myself, I've been using lua vips with plain lua.

@kleisauke, I think you are deploying lua-vips and openresty, do you have a sample config you could share?

[kleisauke]

Sample config: https://github.com/weserv/images/blob/4.x/config/nginx/conf.d/imagesweserv.conf Dockerfile: https://github.com/weserv/images/blob/4.x/Dockerfile

Dockerfile needs some testing/fine-tuning (we are not using Docker on our dedicated server).

[maxim-avramenko]

Sample config: https://github.com/weserv/images/blob/4.x/config/nginx/conf.d/imagesweserv.conf Dockerfile: https://github.com/weserv/images/blob/4.x/Dockerfile

Dockerfile needs some testing/fine-tuning (we are not using Docker on our dedicated server).

awesome! Thanks for your Dockerfile and config examples! Im trying deploy with Ubuntu bionic, and your examples with centOS very usefully