search

caddyserver / caddy

Fast and extensible multi-platform HTTP/1-2-3 web server with automatic HTTPS
https://caddyserver.com
Apache License 2.0
57.85k stars 4.02k forks source link

panic: slice bounds out of range #3623

Closed endelwar closed 4 years ago

endelwar commented 4 years ago

I'm seeing in caddy logs some panics, here is a sample:

Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: 2020/07/17 15:53:12 http: panic serving 207.46.13.85:3472: runtime error: slice bounds out of range [274:272]
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: goroutine 15138437 [running]:
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: net/http.(*conn).serve.func1(0xc000ca9040)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         net/http/server.go:1772 +0x139
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: panic(0x16af0a0, 0xc005399640)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         runtime/panic.go:975 +0x3e3
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp/fileserver.MatchFile.firstSplit(0x17c907d, 0x10, 0xc0003e3400, 0x3, 0x4, 0x0, 0x0, 0xc0003e33c0, 0x1, 0x4, ...)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/fileserver/matcher.go:287 +0x201
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp/fileserver.MatchFile.selectFile(0x17c907d, 0x10, 0xc0003e3400, 0x3, 0x4, 0x0, 0x0, 0xc0003e33c0, 0x1, 0x4, ...)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/fileserver/matcher.go:187 +0xb73
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp/fileserver.MatchFile.Match(0x17c907d, 0x10, 0xc0003e3400, 0x3, 0x4, 0x0, 0x0, 0xc0003e33c0, 0x1, 0x4, ...)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/fileserver/matcher.go:160 +0xc2
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.MatcherSet.Match(0xc000408c50, 0x1, 0x1, 0xc004d1f800, 0x10)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/routes.go:270 +0x68
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.MatcherSets.AnyMatch(0xc00014ac60, 0x1, 0x1, 0xc004d1f800, 0xc006472f60)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/routes.go:291 +0x75
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapRoute.func1.1(0x1ad2d60, 0xc002795340, 0xc004d1f800, 0x0, 0xc00014ab60)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/routes.go:202 +0x68
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc0037205a0, 0x1ad2d60, 0xc002795340, 0xc004d1f800, 0xc002795300, 0xc00617bda0)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/caddyhttp.go:58 +0x44
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapRoute.func1.1(0x1ad2d60, 0xc002795340, 0xc004d1f800, 0xc006473070, 0xc00074c780)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/routes.go:203 +0x2ec
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc0037205c0, 0x1ad2d60, 0xc002795340, 0xc004d1f800, 0x7ff67c9f41f8, 0xc005507500)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/caddyhttp.go:58 +0x44
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp/encode.(*Encode).ServeHTTP(0xc00014afe0, 0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0x1aba840, 0xc0037205c0, 0x0, 0x0)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/encode/encode.go:90 +0x105
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1(0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0xc006473101, 0xc0003e6000)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/routes.go:256 +0x5f
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc005507860, 0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0xc008ff6508, 0x1)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/caddyhttp.go:58 +0x44
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.VarsMiddleware.ServeHTTP(0xc000802c60, 0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0x1aba840, 0xc005507860, 0xc0005c0000, 0x7ff6a36c0e98)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/vars.go:52 +0x2e2
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1(0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0x7ff67c9cd8c8, 0xc000188ef0)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/routes.go:256 +0x5f
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc005507890, 0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0x1, 0x0)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/caddyhttp.go:58 +0x44
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapRoute.func1.1(0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0x1, 0x1)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/routes.go:231 +0x121
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc0037205e0, 0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0x186dda8, 0x1aba840)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/caddyhttp.go:58 +0x44
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Subroute).ServeHTTP(0xc00014a580, 0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0x1aba840, 0x186dda8, 0x16201a0, 0xff1001)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/subroute.go:74 +0x8a
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1(0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0x1ab6080, 0xc00014a580)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/routes.go:256 +0x5f
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc005507800, 0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0xc006473501, 0xff0fa8)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/caddyhttp.go:58 +0x44
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapRoute.func1.1(0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0x0, 0xc0005bec40)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/routes.go:231 +0x121
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc000646660, 0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0x0, 0x18)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/caddyhttp.go:58 +0x44
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapRoute.func1.1(0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0x40d1e8, 0x20)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/routes.go:203 +0x2ec
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc000646680, 0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0xc003720400, 0xc006473678)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/caddyhttp.go:58 +0x44
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Server).enforcementHandler(0xc000676360, 0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0x1aba840, 0xc000646680, 0xb478afad32781, 0x426fcc
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/server.go:277 +0x9b
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Server).wrapPrimaryRoute.func1(0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0xc00426fcc7, 0xb478afad32781)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/server.go:253 +0x5a
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc0006466a0, 0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800, 0xc0054c3400, 0xc004d1f800)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/caddyhttp.go:58 +0x44
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Server).ServeHTTP(0xc000676360, 0x7ff67c9f41f8, 0xc005507500, 0xc004d1f800)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         github.com/caddyserver/caddy/v2@v2.1.1/modules/caddyhttp/server.go:202 +0x4b3
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: net/http.serverHandler.ServeHTTP(0xc000b32a80, 0x1ad5320, 0xc000687420, 0xc005638f00)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         net/http/server.go:2807 +0xa3
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: net/http.(*conn).serve(0xc000ca9040, 0x1adcb20, 0xc0005e39c0)
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         net/http/server.go:1895 +0x86c
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]: created by net/http.(*Server).Serve
Jul 17 15:53:12 webserver.endelwar.it caddy[29602]:         net/http/server.go:2933 +0x35c

this is an extract of the relative access.log:

{
    "level":"info",
    "ts":1594993992.0698013,
    "logger":"http.log.access.log2",
    "msg":"handled request",
    "request":{
        "method":"GET",
        "uri":"/%E2%C3%83%C6%92%C3%86%E2%80%99%C3%83%E2%80%A0%C3%A2%E2%82%AC%E2%84%A2%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%AF%C3%83%C6%92%C3%86%E2%80%99%C3%83%C2%A2%C3%A2%E2%80%9A%C2%AC%C3%85%C2%A1%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%BF%C3%83%C6%92%C3%86%E2%80%99%C3%83%C2%A2%C3%A2%E2%80%9A%C2%AC%C3%85%C2%A1%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%BD%C3%83%C6%92%C3%86%E2%80%99%C3%83%E2%80%A0%C3%A2%E2%82%AC%E2%84%A2%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%AF%C3%83%C6%92%C3%86%E2%80%99%C3%83%C2%A2%C3%A2%E2%80%9A%C2%AC%C3%85%C2%A1%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%BF%C3%83%C6%92%C3%86%E2%80%99%C3%83%C2%A2%C3%A2%E2%80%9A%C2%AC%C3%85%C2%A1%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%BD",
        "proto":"HTTP/1.1",
        "remote_addr":"207.46.13.85:3472",
        "host":"www.endelwar.it",
        "headers":{
            "Cache-Control":[
                "no-cache"
            ],
            "Connection":[
                "Keep-Alive"
            ],
            "Pragma":[
                "no-cache"
            ],
            "Accept":[
                "*/*"
            ],
            "Accept-Encoding":[
                "gzip, deflate"
            ],
            "User-Agent":[
                "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
            ]
        },
        "tls":{
            "resumed":false,
            "version":771,
            "ciphersuite":49196,
            "proto":"",
            "proto_mutual":true,
            "server_name":"www.endelwar.it"
        }
    },
    "common_log":"207.46.13.85 - - [17/Jul/2020:15:53:12 +0200] \"GET /%E2%C3%83%C6%92%C3%86%E2%80%99%C3%83%E2%80%A0%C3%A2%E2%82%AC%E2%84%A2%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%AF%C3%83%C6%92%C3%86%E2%80%99%C3%83%C2%A2%C3%A2%E2%80%9A%C2%AC%C3%85%C2%A1%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%BF%C3%83%C6%92%C3%86%E2%80%99%C3%83%C2%A2%C3%A2%E2%80%9A%C2%AC%C3%85%C2%A1%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%BD%C3%83%C6%92%C3%86%E2%80%99%C3%83%E2%80%A0%C3%A2%E2%82%AC%E2%84%A2%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%AF%C3%83%C6%92%C3%86%E2%80%99%C3%83%C2%A2%C3%A2%E2%80%9A%C2%AC%C3%85%C2%A1%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%BF%C3%83%C6%92%C3%86%E2%80%99%C3%83%C2%A2%C3%A2%E2%80%9A%C2%AC%C3%85%C2%A1%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%BD HTTP/1.1\" 0 0",
    "duration":0,
    "size":0,
    "status":0,
    "resp_headers":{
        "Server":[
            "Caddy"
        ]
    }
}

Other URIs that trigger the panic are these:

/%E2%C3%83%C6%92%C3%86%E2%80%99%C3%83%E2%80%9A%C3%82%C2%AF%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%BF%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%BD%C3%83%C6%92%C3%86%E2%80%99%C3%83%E2%80%9A%C3%82%C2%AF%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%BF%C3%83%C6%92%C3%A2%E2%82%AC%C5%A1%C3%83%E2%80%9A%C3%82%C2%BD

/\ufffd½\ufffd\ufffdļ\ufffd\ufffd\ufffd.rar

/\ufffd½\ufffd\ufffdļ\ufffd\ufffd\ufffd.zip

/\ufffd½\ufffd\ufffdļ\ufffd\ufffd\ufffd.tar.gz

I've found both GET and HEAD requests, coming from bing search and a webscraper looking for specific archive files in webroot

This happens on a debian 9 with Caddy 2.1.1 with no additional modules.

Caddyfile is the following:

{
    email my@email.test
    on_demand_tls {
        ask https://mydomain.test/api/allowed-domain
    }
}

(global-encode) {
    encode zstd gzip
}

http:// {
    import global-encode
    root * /srv/www/sat-checker/current/public/
    file_server
    php_fastcgi unix//run/php/php7.2-fpm.sock
    log {
        output file /var/log/caddy/sat-checker.log
    }
}

# HTTPS catch-all with on-demand SSL cert
https:// {
    import global-encode
    tls {
        on_demand
    }

    root * /srv/www/sitisatellite/current/public/

    @urlblock {
        path_regexp wpattack /(wp-admin|wp-login|wp-content|xmlrpc|wp|wordpress)
    }
    respond @urlblock 410

    file_server
    php_fastcgi unix//run/php/php7.2-fpm.sock

    log {
        output file /var/log/caddy/sitisatellite.log
    }
}
mholt commented 4 years ago

Thanks for the great report. I was able to reproduce the bug, so finding a fix will be much easier with the information you provided.

mholt commented 4 years ago

This is caused because the {http.request.uri.path} placeholder uses the req.URL.Path value. Using req.URL.RawPath (or req.URL.EscapedPath()) fixes it, but this is the un-decoded form of the URL (i.e. Path is unescaped, and RawPath is escaped).

mholt commented 4 years ago

So this is fun: it turns out len(str) != len(strings.ToLower(str)) https://play.golang.org/p/DrkIBkXcfUq

Still looking for a fix without changing the placeholder (since I think most users want the path placeholder to return the unescaped path).

mholt commented 4 years ago

Okay, so the std lib has strings.EqualFold() which does case-insensitive string comparison, but there's no case-insensitive substring search (i.e. strings.IndexFold()) which is really what we need here.

So I wrote my own, and now the tests pass (including a new one for this issue), so I hope that means it's correct:

func indexFold(haystack, needle string) int {
    nlen := len(needle)
    for i := 0; i+nlen < len(haystack); i++ {
        if strings.EqualFold(haystack[i:i+nlen], needle) {
            return i
        }
    }
    return -1
}

This should fix the problem because before we were finding a position using the lower-cased version of the path, then using that same position with the original version of the path, which has a different length (😬)... this solution does not create a lowercase version of the path, but still does case-insensitive substring searching.

mholt commented 4 years ago

Should be fixed now, please give it a try with the latest build artifacts!

endelwar commented 4 years ago

I've got zero new panic in the last 10 days of tests, fix is working as expected, thaks!