ERROR: cqueue: attempt to yield across metamethod/C-call boundary

[charlesboyo]

Hello again.

I have implemented a small HTTP client based on the "simple client example showing SSL and cqueue controller nesting" at http://25thandclement.com/~william/projects/cqueues.html.

I modified that to parse the headers and body, then decode the body as JSON and optionally invoke a command line function (using io.popen) which then returns its output was another cq:wrap function that does a POST to the server.

Runs beautifully, but every few runs I get the following error:

lua: http_test.lua:134: attempt to yield across metamethod/C-call boundary stack traceback: C: in function 'assert' http_test.lua:134: in main chunk

How can I get more information? And is there anything wrong with the process I have described above that suggests it won't work? I have to confess I am pretty new to Lua and haven't fully understood the C/Lua interactions or the cqueues functionality.

Regards,

Charles

[charlesboyo]

Following is my code:

local cqueues = require("cqueues")
local socket = require("cqueues.socket")

local host, port, timeout = ...

host = host or "10.0.15.6"
port = tonumber(port or 5000)
timeout = tonumber(timeout or 5)

local cq = cqueues.new()

cq:wrap(function()
    while true do
        local http = socket.connect(host, port)

        http:write("GET / HTTP/1.0\n")
        http:write(string.format("Host: %s:%d\n", host, port))
        http:write(string.format("Prefer: wait=%d\n\n", timeout))

        local status = http:read()
        local http_ver, status_code, status_text = string.match(status, "(HTTP/%d%.%d)%s(%d+)%s(.+)")
        print('\nGET', status_code, status_text)

        local headers, body = { }, { }
        for ln in http:lines() do
            if ln == '' then break end
            table.insert(headers,ln)
        end

        for ln in http:lines() do
            print(ln)
            table.insert(body,ln)
        end

        print("--http.close()--")
        http:close()

    end
end)

assert(cq:loop())

and the resulting output:

# lua mini_http.lua

GET     200     OK
{ "code": 0, "timeout": 5, "wait": 1, "command": ["WGjS1SVmfQtdJb4R","NOOP"] }
--http.close()--

GET     200     OK
{ "code": 0, "timeout": 5, "wait": 0, "command": ["jpgjVvzy59BVxdju","NOOP"] }
--http.close()--

GET     200     OK
{ "code": 0, "timeout": 5, "wait": 2, "command": ["uFvu5DsEnqAV4vqV","NOOP"] }
lua: mini_http.lua:44: attempt to yield across metamethod/C-call boundary
stack traceback:
        [C]: in function 'assert'
        mini_http.lua:44: in main chunk
        [C]: ?

[daurnimator]

lua: mini_http.lua:44: attempt to yield across metamethod/C-call boundary

There's gotta be more to it: the script you pasted isn't 44 lines long.

That error message implies that you tried to yield in a place it isn't allowed. These rules change depending on your lua version, so please share that too.

[charlesboyo]

# lua -v
Lua 5.1.5  Copyright (C) 1994-2012 Lua.org, PUC-Rio

The full code:

local cqueues = require("cqueues")
local socket = require("cqueues.socket")

local host, port, timeout = ...

host = host or "10.0.15.6"
port = tonumber(port or 5000)
timeout = tonumber(timeout or 5)

local cq = cqueues.new()

cq:wrap(function()
    while true do
        local http = socket.connect(host, port)
        if port == 443 then
            http:starttls()
        end

        http:write("GET / HTTP/1.0\n")
        http:write(string.format("Host: %s:%d\n", host, port))
        http:write(string.format("Prefer: wait=%d\n\n", timeout))

        local status = http:read()
        local http_ver, status_code, status_text = string.match(status, "(HTTP/%d%.%d)%s(%d+)%s(.+)")
        print('\nGET', status_code, status_text)

        local headers, body = { }, { }
        for ln in http:lines() do
            if ln == '' then break end
            table.insert(headers,ln)
        end

        for ln in http:lines() do
            table.insert(body,ln)
            print(ln)
        end

        print("--http.close()--")
        --http:close()

    end
end)

assert(cq:loop())

The output:

# lua mini_http.lua

GET     200     OK
{ "code": 0, "timeout": 5, "wait": 3, "command": ["9TYyPDCLEKCEdUZx","NOOP"] }
--http.close()--

GET     200     OK
{ "code": 0, "timeout": 5, "wait": 4, "command": ["zcZd1Wbxv3ApRZQx","NOOP"] }
--http.close()--

GET     200     OK
{ "code": 0, "timeout": 5, "wait": 1, "command": ["KQ3SmVyXW41LRp5P","NOOP"] }
--http.close()--

GET     200     OK
{ "code": 0, "timeout": 5, "wait": 3, "command": ["Psrurxlwh2hkMzuR","NOOP"] }
lua: mini_http.lua:44: attempt to yield across metamethod/C-call boundary
stack traceback:
        [C]: in function 'assert'
        mini_http.lua:44: in main chunk
        [C]: ?

[daurnimator]

The issue is the for ln in http:lines() do. Lua 5.1 doesn't allow yielding from inside of for loop iterators.

[charlesboyo]

There are two of those:

    local headers, body = { }, { }
    for ln in http:lines() do
        if ln == '' then break end
        table.insert(headers,ln)
    end

    for ln in http:lines() do
        table.insert(body,ln)
        print(ln)
    end

What exactly is doing the "yielding", how can I resolve this?

[daurnimator]

What exactly is doing the "yielding", how can I resolve this?

1. Upgrade to Lua 5.2 or above

Alternatively,

2. Write them in non-for-loop form. e.g.

   local iter, state, last = http:lines()
   while true do
   local ln = iter(state, last)
   if not ln then break end
   .......
   end

or more simply for :lines():

while true do
    local ln = http:read()
    if not ln then break end
    .......
end