Image load sometimes fails

[dalini]

I've running info-beamer on a pi and fetch a remote image via cron and then render it within the node. Sometimes the image can't be loaded (eventually while written?). Sometimes the system recovers itself and shows the image again. Sometimes it stays in this mode.

So how do I prevent this - can I catch this and clean up somehow? So next cycle works again or keep the old image in case the new one can not be loaded?

Following setup:

cronjob: */1 * * * * wget http://www.mydomain/image.jpg -O /home/pi/info-beamer-pi/info-beamer-nodes/mytest/image.jpg

node.lua:

gl.setup(NATIVE_WIDTH, NATIVE_HEIGHT)
local background = resource.load_image("image.jpg")

util.file_watch("image.jpg", function(content)
    background = resource.load_image("image.jpg")
    return background
end)

function node.render()
    gl.clear(0, 0.423, 0.694, 1)
    background:draw(110,550,WIDTH-100,HEIGHT-150, 0.8)
end

Normal behavior output:

[mytest] update +image.jpg (write closed)
[image.c] loading mytest/image.jpg
[image.c] mytest/image.jpg is 640x480

Error behavior output:

[mytest] update +image.jpg (write closed)
[image.c] loading mytest/image.jpg
[image.c] mytest/image.jpg is 640x480
glGetError 0x505

Sometimes I get in addition:

[mytest] runtime error: mytest/node.lua:17: cannot render child to image: out of memory
stack traceback:
        [C]: in function 'render_child'
        mytest/node.lua:17: in function <mytest/node.lua:10>
        bundled userlib.lua:616: in function <bundled userlib.lua:613>
glGetError 0x505
glGetError 0x505
glGetError 0x505

[mytest] runtime error: mytest/node.lua:14: image loaders failed to parse this image format
stack traceback:
        [C]: in function 'draw'
        mytest/node.lua:14: in function <mytest/node.lua:10>
        bundled userlib.lua:616: in function <bundled userlib.lua:613>

[sheepherder]

You need to replace the file atomically. Download it to image.jpg.tmp and then do

mv image.jpg.tmp image.jpg

[dividuum]

While @sheepherder is correct with the atomic replacement, I don't think that's the problem here (see below for more info on that). It looks like info-beamer is running out of GPU memory. This can happen if you don't clean up unused image objects. See https://info-beamer.com/lnk/gc for more information about this. The quick (and dirty) solution would be to set node.set_flag("slow_gc", false) somewhere at the top of your node.lua script.

What you're trying to archive can also be solved much easier by just using util.resource_loader:

util.resource_loader{
    "background.jpg"
}

This helper function will automatically keep the global variable 'background' in sync with the file 'background.jpg'.

Regarding atomic replacement of files: Normally it's not required to write to a temporary file and rename: info-beamer only listens for the close-write event, so it will only pick up files that are completely written. If you can guarantee that you always write complete files, you'll be fine. If that might not be the case, it's better to create a temporary file and rename that once it's complete.

[sheepherder]

I suspected the problem to be an incomplete file, because I ran into that problem (I think) on the exact same day. In my case though, it was a json file written by an helper python script.

I use a bash script to start the helper in the background and info-beamer at the same time. In around half the cases, info-beamer failed to pick up the json on start correctly, probably because it was written at the same time. The write is just a simple json.dump of a single variable to a file in one line of python, but obviously the timing was bad. Changing to writing to an temp file and an atomic rename seemed to fix the problem.

Starting up info-beamer and associated scripts at the same time seems to be a normal use case for me. So I suspect atomic renames are a must even though you listen to the close event.

A proposed fix would be to check in info-beamer on first read of files, whether the file is open for writing somewhere else (don't know if that's possible) and delay the read for a short time if that's the case. Or did I run into a completely different problem and my analysis is wrong?

[dividuum]

Thanks @sheepherder for that information. You analysis is correct. Atomic renames are required for your usecase. I'm also not sure if it's possible to check "file usage". I'm fairly certain there's no race free way to do this. So a rename is the way to go. I'll add some notice to the documentation.

[dividuum]

@sheepherder, documentation is updated: https://info-beamer.com/doc/info-beamer#node.event/content_update

[dalini]

The change with the resource_loader already improved the situation significantly! Even without the atomic mv - which will make it even more robust! ;)