openresty / lua-nginx-module

Embed the Power of Lua into NGINX HTTP servers
https://openresty.org/
11.34k stars 2.03k forks source link

errlog get_logs doesnt get error logs in privileged agent and visa versa #2336

Closed Rockybilly closed 3 months ago

Rockybilly commented 4 months ago

errlog module and lua_capture_error_log fails to capture logs in workers when called from a privileged agent, and fails to get logs in privileged agent when called from worker

Here is an example config to reproduce, tried with 1.21.4.1 and 1.25.3.1

daemon off;
master_process on;
user root;

error_log /dev/stdout info;

events {

}

http {
    lua_capture_error_log 100k;

    init_by_lua_block {

        local process = require "ngx.process"
        local ok, err = process.enable_privileged_agent()
        if not ok then
            ngx.log(ngx.ERR, "enables privileged agent failed error:", err)
        end
    }

    init_worker_by_lua_block {
        local process = require "ngx.process"
        ngx.log(ngx.CRIT, "process type: ", process.type())
    }

    server {
        listen 80;

        location / {
            content_by_lua_block {
                local errlog = require("ngx.errlog")
                local res, err = errlog.get_logs() 

                local body = ""
                for i = 1, #res, 3 do
                    local level = res[i]
                    if not level then
                        break
                    end
                    local time = res[i + 1]
                    local msg  = res[i + 2]

                    body = body .. msg .. "\n"
                end

                ngx.say(body)
            }
        }
    }
}

stdout: image

get_logs output in content_by_lua_block image

As you can see, the global error buffer is missing entries. I didn't know if the fault lay with errlog or process modules so I opened the issue here.

Cheers.

zhuizhuhaomeng commented 4 months ago

The error log is directly written to the stderr before the error log is initialized. ngx.errlog can only get the error log after it is initialized.

zhuizhuhaomeng commented 4 months ago

errlog module and lua_capture_error_log fails to capture logs in workers when called from a privileged agent, and fails to get logs in privileged agent when called from worker

Each process only gets its error logs.

Rockybilly commented 4 months ago

errlog module and lua_capture_error_log fails to capture logs in workers when called from a privileged agent, and fails to get logs in privileged agent when called from worker

Each process only gets its error logs.

Ah, this i didn't know. I thought the error logs was gathered eventually in a shared memory from all workers to be collected from any of the workers. Then it is not related to privileged agent at all, but just being different workers is enough.

I should say though, throughout my development, i read errlog.md multiple times and failed to realize this fact. Either it is not mentioned or is not mentioned enough :)

Thank you for your reply.