haproxytech / haproxy-lua-http

Simple Lua HTTP helper && client for use with HAProxy.
Apache License 2.0
56 stars 23 forks source link

A bogus APPCTX [0x7f1a30024b20] is spinning at 100000 calls per second and refuses to die, aborting now! #5

Closed TimWolla closed 4 years ago

TimWolla commented 4 years ago

Reproduction steps:

crasher.cfg

    global
        lua-prepend-path ./?.lua
        lua-load ./crasher.lua

    listen fe1
        mode http
        bind *:8080

        http-request lua.crasher

crasher.lua

local http = require('http')

core.register_action("crasher", { "http-req" }, function(txn)
    local response, err = http.head {
        url = "http://127.0.0.1:9000",
        headers = {},
    }
end)

haproxy -vv

HA-Proxy version 2.3-dev0-950954-25 2020/07/10 - https://haproxy.org/
Status: development branch - not safe for use in production.
Known bugs: https://github.com/haproxy/haproxy/issues?q=is:issue+is:open
Running on: Linux 4.4.0-185-generic #215-Ubuntu SMP Mon Jun 8 21:53:19 UTC 2020 x86_64
Build options :
  TARGET  = linux-glibc
  CPU     = generic
  CC      = gcc
  CFLAGS  = -O2 -g -Wall -Wextra -Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wtype-limits
  OPTIONS = USE_LUA=1 HLUA_PREPEND_PATH=/usr/share/lua/5.2/?.lua

Feature list : +EPOLL -KQUEUE +NETFILTER -PCRE -PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +BACKTRACE -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H +GETADDRINFO -OPENSSL +LUA +FUTEX +ACCEPT4 -ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=4).
Built with Lua version : Lua 5.3.1
Built with network namespace support.
Built without compression support (neither USE_ZLIB nor USE_SLZ are set).
Compression algorithms supported : identity("identity")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built without PCRE or PCRE2 support (using libc's regex instead)
Encrypted password support via crypt(3): yes
Built with gcc compiler version 5.4.0 20160609

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
            fcgi : mode=HTTP       side=BE        mux=FCGI
       <default> : mode=HTTP       side=FE|BE     mux=H1
              h2 : mode=HTTP       side=FE|BE     mux=H2
       <default> : mode=TCP        side=FE|BE     mux=PASS

Available services : none

Available filters :
    [SPOE] spoe
    [COMP] compression
    [TRACE] trace
    [CACHE] cache
    [FCGI] fcgi-app

List of steps:

  1. haproxy -d -f ./crasher.cfg
  2. printf "foo" |nc -l 127.0.0.1 9000
  3. curl localhost:8080
[NOTICE] 192/134456 (17264) : haproxy version is 2.3-dev0-950954-25
[NOTICE] 192/134456 (17264) : path to executable is *snip*/haproxy
[ALERT] 192/134456 (17264) : A bogus APPCTX [0xa6d090] is spinning at 175268 calls per second and refuses to die, aborting now! Please report this error to developers [strm=0xa6d320 src=<LUA_TCP> fe=LUA-SOCKET be=LUA-SOCKET dst=LUA-TCP-CONN rqf=848000 rqa=0 rpf=8000c020 rpa=0 sif=EST,204048 sib=EST,280058 af=0xa6d090,0 csf=(nil),0 ab=(nil),0 csb=0xa6de60,9000 cof=(nil),0:NONE((nil))/NONE((nil))/NONE(0) cob=0xa6a620,42300:PASS(0xa6deb0)/RAW((nil))/tcpv4(16) filters={}]

Thread 1 "haproxy" received signal SIGABRT, Aborted.
0x00007ffff6faa438 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
54  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) t a a bt

Thread 4 (Thread 0x7fffef445700 (LWP 17271)):
#0  0x00007ffff707cad3 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84
#1  0x000000000041da40 in _do_poll (p=<optimized out>, exp=0, wake=0) at src/ev_epoll.c:195
#2  0x00000000005242d7 in run_poll_loop () at src/haproxy.c:3003
#3  0x00000000005246ef in run_thread_poll_loop (data=<optimized out>) at src/haproxy.c:3121
#4  0x00007ffff757d6ba in start_thread (arg=0x7fffef445700) at pthread_create.c:333
#5  0x00007ffff707c4dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 3 (Thread 0x7ffff0047700 (LWP 17270)):
#0  0x00007ffff707cad3 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84
#1  0x000000000041da40 in _do_poll (p=<optimized out>, exp=0, wake=0) at src/ev_epoll.c:195
#2  0x00000000005242d7 in run_poll_loop () at src/haproxy.c:3003
#3  0x00000000005246ef in run_thread_poll_loop (data=<optimized out>) at src/haproxy.c:3121
#4  0x00007ffff757d6ba in start_thread (arg=0x7ffff0047700) at pthread_create.c:333
#5  0x00007ffff707c4dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 2 (Thread 0x7ffff0848700 (LWP 17269)):
#0  0x00007ffff707cad3 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84
#1  0x000000000041da40 in _do_poll (p=<optimized out>, exp=0, wake=0) at src/ev_epoll.c:195
#2  0x00000000005242d7 in run_poll_loop () at src/haproxy.c:3003
#3  0x00000000005246ef in run_thread_poll_loop (data=<optimized out>) at src/haproxy.c:3121
#4  0x00007ffff757d6ba in start_thread (arg=0x7ffff0848700) at pthread_create.c:333
#5  0x00007ffff707c4dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thread 1 (Thread 0x7ffff7fcc1c0 (LWP 17264)):
#0  0x00007ffff6faa438 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff6fac03a in __GI_abort () at abort.c:89
#2  0x00000000004ab979 in stream_dump_and_crash (obj=obj@entry=0xa6d090, rate=175268) at src/stream.c:2691
#3  0x000000000056442e in task_run_applet (t=0xa6d1a0, context=0xa6d090, state=<optimized out>) at src/applet.c:98
#4  0x0000000000567a22 in run_tasks_from_lists (budgets=budgets@entry=0x7fffffffd9f0) at src/task.c:477
#5  0x0000000000568549 in process_runnable_tasks () at src/task.c:673
#6  0x000000000052431c in run_poll_loop () at src/haproxy.c:2956
#7  0x00000000005246ef in run_thread_poll_loop (data=data@entry=0x0) at src/haproxy.c:3121
#8  0x000000000041b5ed in main (argc=<optimized out>, argv=0x7fffffffdef8) at src/haproxy.c:3819

see also: https://github.com/TimWolla/haproxy-auth-request/runs/860809162?check_suite_focus=true

TimWolla commented 4 years ago

This appears to be a bug in HAProxy itself: haproxy/haproxy#744