Can't get rid of "Queue is broken, delay not a dpos" error message

[GoogleCodeExporter]

What steps will reproduce the problem?

I use a lua config file that will spawn programs (that will run for severals 
seconds) and then copy intermediate files to a remote server using sftp.

What is the expected output? What do you see instead?

Everything works correctly but at the end lsyncd crashes saying :
.....
00:55:24 Call: getAlarm()
00:55:24 Debug: getAlarm returns: (false)
00:55:24 Masterloop: going into select (no timeout).
00:55:24 Call: collectProcess()
00:55:24 Delay: collected an event
00:55:24 Delay: Finish of Create on /home/xxxx
00:55:24 Call: cycle()
00:55:24 Function: invokeActions('Sync1',(Timestamp: 42966265.74))
00:55:24 Call: getAlarm()
00:55:24 Debug: getAlarm returns: (false)
00:55:24 Masterloop: going into select (no timeout).
00:55:24 Call: collectProcess()
00:55:24 Delay: collected an event
00:55:24 Error: IN LUA: lsyncd.lua:1150: Queue is broken, delay not a dpos
00:55:24 Error: Backtrace 1 :[C]:-1
00:55:24 Error: Backtrace 2 :lsyncd.lua:1150
00:55:24 Error: Backtrace 3 :lsyncd.lua:1207
00:55:24 Error: Backtrace 4 :lsyncd.lua:2500

What version of the product are you using? On what operating system?

lsyncd 2.0.2 on ubuntu 64 bits 10.04

Please provide any additional information below.

I write my conf file using the image magic example. No clue at the moment.
I do not understand the error, what does mean the error message ? Lsyncd tries 
to unstack a delay event that does not exist ?
Thank you in advance for looking into this issue

Original issue reported on code.google.com by pierre.a...@gmail.com on 8 Feb 2011 at 12:05

[GoogleCodeExporter]

Any error printing a backtrace, originating from lsyncd.lua is very likely a 
coding bug. Can you by chance send me your config complete file?

Original comment by axk...@gmail.com on 8 Feb 2011 at 6:08

[GoogleCodeExporter]

Hello,

you'll find my conf file hereunder :

synchro = {
    maxProcesses = 99,
        delay        = 1,

    action = function(inlet)
        -- Get current event
        local event = inlet.getEvent()

        if event.isdir then
                        -- ignores events on dirs
                        inlet.discardEvent(event)
                        return
                end

        -- Get source information
        local p = event.sourcePathname
        local dest = event.target
        local wdir = event.source..'../test_folder/'
        local ext = string.match(p, ".*%.([^.]+)$")
        local base = string.match(p, "(.*)%.[^.]+$")

        if event.etype == "Create" then
            -- build one bash command
            local cmd = ""

            cmd = cmd..'cp "'..p..'" '..wdir
            log("Copying to working directory: "..wdir)
                        spawnShell(event, cmd)          
            cmd = ""
            local pp = wdir..event.pathname

            cmd = cmd..
                '/usr/bin/btmakemetafile "'..
                pp..'" http://12.0.0.6:1080/announce'

            log("Creating torrent file from: "..pp)
            spawnShell(event, cmd)

            cmd = ""

            cmd = cmd..
                '/usr/bin/scp "'..pp..'.torrent" '..dest
            log("Sending torrent file : "..pp..'.torrent')
                        spawnShell(event, cmd)
            --cmd = ""
            --cmd = cmd..
            --  '/usr/bin/setsid /usr/bin/btdownloadheadless "'..
            --  pp..'.torrent"'
            --log("Seeding : "..pp..'.torrent')
                        --spawnShell(event, cmd)
            return
        end
        inlet.discardEvent(event)
    end,

}

sync{synchro, source="/home/px/scratchpad/no-cloud/watched_folder",
target="12.0.0.34:/home/px/scratchpad/no-cloud/incoming_folder",
action = action, delay = 1, init = false}

Original comment by pierre.a...@gmail.com on 8 Feb 2011 at 9:09

[GoogleCodeExporter]

Okay, you cannot spawn multiple processes on the same event. Note that
spawn() and spawnShell() do not wait for the process to finish, Lsyncd
will start the process and continue working right away.Lsyncd has one
process on one event. Otherwise it wouldn't know what do if for
example one of them failed. Also I suppose in your case you want them
to be executed sequentally anyway.

So just chain your commands into one Shell call and you'll be fine:
                      local cmd = ""
                      cmd = cmd..'cp "'..p..'" '..wdir
                      local pp = wdir..event.pathname
                      cmd = cmd.." && "..
                              '/usr/bin/btmakemetafile "'..
                              pp..'" http://12.0.0.6:1080/announce'
                      cmd = cmd.. " && " ..
                              '/usr/bin/scp "'..pp..'.torrent" '..dest
                      log("Invoking processing of : ",pp,'.torrent')
                      spawnShell(event, cmd)

Note that the '..' is not too efficient since the garbage collection
of string parts it uses. When using it a lot following is faster and
in my opinion easier to read:

     local pp = wdir..event.pathname
     local cmdt = {
          'cp "', p, '" ', wdir, " &&\n",
          '/usr/bin/btmakemetafile "', pp, '
"http://12.0.0.6:1080/announce', ' &&\n',
          '/usr/bin/scp "', pp, '.torrent" ', dest }
     log("Invoking processing of : ",pp,'.torrent')
     spawnShell(event, table.concat(cmdt))

Despite this, its a Lsyncd bug it doesnt catch the multiple spawns on
the same event when they are happening, I'll fix that. What raised the
aboth error was the event being unqued by the first spawn and the
second one catching an internal error, since it tries to unque
something that isn't queued.

Original comment by axk...@gmail.com on 8 Feb 2011 at 10:31

[GoogleCodeExporter]

Thank you, it worked without any more problem.
P.Alex

Original comment by pierre.a...@gmail.com on 8 Feb 2011 at 10:13

[GoogleCodeExporter]

Original comment by axk...@gmail.com on 24 Feb 2011 at 3:50

• Changed state: Fixed