I have collected some of the configs that I have encountered during developing #192. These are mostly about testing the matching and routing logic of layer4. Maybe they can some day be used to setup an automated testing suite. For now this is just intended as reference, so they are easier to find and maybe help other developers.
prefetch
This tests basic functionality of multiple routes that are executed one after another. The interesting bit is that curl does not send the http2 request in one go. But rather first the http2 preface followed by the rest of the http2 request in a different flush (see https://github.com/mholt/caddy-l4/pull/72#issuecomment-1322798286). So usually multiple prefetch calls are needed.
All should succeed and show "Hello World" in the response.
socks4 before socks5
This tests if matchers that are tried first and require more data than the client has sent will block all other matchers.
This was part of the initial motivation for #192.
The socks4 command will likely fail for you, except you are connected to Mullvad VPN.
But the socks5 one should print your external ip address.
What should not happen is that the socks5 command blocks indefinitely like before #192.
subroute
This tests that the subroute handler works after another matcher was already executed.
This is a reproducer for a bug introduced by #192.
This tests if it is possible to use tls inside tls. Which is for example used by Signal proxies.
Or more generally if the route order in the config is still respected.
This is a reproducer for a bug introduced by #192.
This is a reproducer for a bug that was introduced in #196.
It was triggered by a non terminal handler that did not consume its bytes and would thus be matched again in the next matching loop iteration. Normally a call to prefetch should have happened after the first loop iteration, which blocks the endless loop until timeout.
In this example config a proxy_protocol handler is used that does not accept the proxy protocol send by the test command and thus the matched bytes are not consumed.
Should fail after the matching_timeout (by default 3s) with a "aborted matching according to timeout" error in the caddy log.
If the bug is triggered a caddy thread will run at 100% percent cpu load indefinitely and no error or connection stats are visible in the caddy log.
listener wrapper fallback
This tests if it is possible to use the layer4 listener wrapper with caddy as fallback. Meaning connections not matched by layer4 are passed back to the caddy http app.
This is a reproducer for a bug that was introduced with #192, that was fixed by #208.
The first (http) request should contain "Hello World from testing caddy" in the response.
The second one (https) request should contain "Hello World from backend caddy" in the response.
listener wrapper matcher with no handler
This tests if it is possible to use a route with a matcher but without any handlers to workaround #207 in most cases.
The first (http) request should contain "Hello World from testing caddy" in the response.
The second one (https) request should contain "Hello World from backend caddy" in the response.
At the time of writing this works.
matcher without io
This tests if it is possible to use a route with a matcher that does not read from the connection.
Testing commands:
Simulating a server that sends data after connection establishment.
echo "Hello World" | nc -v -l 127.0.0.1 10080
Connecting to caddy as client
nc -v 127.0.0.1 10443
You should see "Hello World" being printed on the client shell. You have to restart the server nc command each time.
route without matchers after matcher
This tests if routes without matchers are not preferred when there exist matching routes before them.
This is a reproducer for a regression introduced in #208, that was fixed by #239.
The first curl should print "Hello from backend" and the second "Hello from fallback". During the regression both returned "Hello from fallback".
double matching not allowed
This tests if it is possible to match the same route multiple times. For example by recursively using tls in tls. This should not be possible, because it is confusing and often not what users want. But it was possible on some commits between #192 and #208.
The test program should print an EOF error since layer4 should immediately force close the connection after the first tls layer is processed. But on 6a8be7c4b8acb0c531b6151c94a9cd80894acce1 for example it would print "Hello World".
out of order matching not allowed
This tests if out of order route matching is possible. It should not be possible, because it is confusing and often not what users want. Instead it should only be possible to execute routes in the order they are defined in the config. For example this config is using a proxy protocol handler as second route after tls.
This should immediately fail with layer4 closing the connection, because the proxy protocol route is not the first one. Or in other words the tls route should be unmatchable after the proxy protocol route was executed.
I have collected some of the configs that I have encountered during developing #192. These are mostly about testing the matching and routing logic of layer4. Maybe they can some day be used to setup an automated testing suite. For now this is just intended as reference, so they are easier to find and maybe help other developers.
prefetch
This tests basic functionality of multiple routes that are executed one after another. The interesting bit is that curl does not send the http2 request in one go. But rather first the http2 preface followed by the rest of the http2 request in a different flush (see https://github.com/mholt/caddy-l4/pull/72#issuecomment-1322798286). So usually multiple prefetch calls are needed.
References:
prefetch.json
```json { "admin": { "disabled": true }, "logging": { "logs": { "default": {"level":"DEBUG", "encoder": {"format":"console"}} } }, "apps": { "tls": { "certificates": { "automate": ["localhost"] }, "automation": { "policies": [{ "subjects": ["localhost"], "issuers": [{ "module": "internal" }] }] } }, "layer4": { "servers": { "https": { "listen": ["0.0.0.0:10443"], "routes": [ { "match": [ {"proxy_protocol": {}} ], "handle": [ { "handler": "proxy_protocol", "allow": ["0.0.0.0/0"] } ] }, { "match": [ {"tls": {"sni": ["localhost"]}} ], "handle": [ {"handler": "tls"} ] }, { "match": [ {"http": [{"host": ["localhost"]}]} ], "handle": [ { "handler": "proxy", "upstreams": [{"dial": ["127.0.0.1:10080"]}] } ] } ] } } }, "http": { "servers": { "backend": { "protocols": ["h1","h2","h2c"], "listen": ["127.0.0.1:10080"], "routes": [ { "handle": [{ "handler": "static_response", "status_code": "200", "body": "Hello World\n", "headers": { "Content-Type": ["text/plain"] } }] } ] } } } } } ```Testing commands:
All should succeed and show "Hello World" in the response.
socks4 before socks5
This tests if matchers that are tried first and require more data than the client has sent will block all other matchers. This was part of the initial motivation for #192.
References:
socks4-before-socks5.json
```json { "admin": { "disabled": true }, "logging": { "logs": { "default": {"level":"DEBUG", "encoder": {"format":"console"}} } }, "apps": { "layer4": { "servers": { "https": { "listen": ["0.0.0.0:1080"], "routes": [ { "match": [ {"socks4": {}} ], "handle": [ { "handler": "proxy", "upstreams": [{"dial": ["10.64.0.1:1080"]}] } ] }, { "match": [ {"socks5": {}} ], "handle": [ {"handler": "socks5"} ] } ] } } } } } ```Testing commands:
The socks4 command will likely fail for you, except you are connected to Mullvad VPN. But the socks5 one should print your external ip address. What should not happen is that the socks5 command blocks indefinitely like before #192.
subroute
This tests that the subroute handler works after another matcher was already executed. This is a reproducer for a bug introduced by #192.
References:
subroute.json
```json { "admin": { "disabled": true }, "logging": { "logs": { "default": {"level":"DEBUG", "encoder": {"format":"console"}} } }, "apps": { "tls": { "certificates": { "automate": ["localhost"] }, "automation": { "policies": [{ "subjects": ["localhost"], "issuers": [{ "module": "internal" }] }] } }, "layer4": { "servers": { "https": { "listen": ["0.0.0.0:10443"], "routes": [ { "match": [ {"tls": {}} ], "handle": [ { "handler": "subroute", "routes": [ { "match": [ {"tls": {"sni": ["localhost"]}} ], "handle": [ {"handler": "tls"}, { "handler": "proxy", "upstreams": [{"dial": ["127.0.0.1:10080"]}] } ] } ] } ] } ] } } }, "http": { "servers": { "backend": { "protocols": ["h1","h2","h2c"], "listen": ["127.0.0.1:10080"], "routes": [ { "handle": [{ "handler": "static_response", "status_code": "200", "body": "Hello World\n", "headers": { "Content-Type": ["text/plain"] } }] } ] } } } } } ```Testing commands:
Should contain "Hello World" in the response.
tls in tls
This tests if it is possible to use tls inside tls. Which is for example used by Signal proxies. Or more generally if the route order in the config is still respected. This is a reproducer for a bug introduced by #192.
References:
tls-in-tls.json
```json { "admin": { "disabled": true }, "logging": { "logs": { "default": {"level":"DEBUG", "encoder": {"format":"console"}} } }, "apps": { "tls": { "certificates": { "automate": ["outer", "inner"] }, "automation": { "policies": [ { "subjects": ["outer"], "issuers": [{ "module": "internal" }] }, { "subjects": ["inner"], "issuers": [{ "module": "internal" }] } ] } }, "layer4": { "servers": { "https": { "listen": ["0.0.0.0:10443"], "routes": [ { "handle": [ {"handler": "tls"} ] }, { "match": [ {"tls": {"sni": ["inner"]}} ], "handle": [ {"handler": "tls"}, { "handler": "proxy", "upstreams": [{"dial": ["127.0.0.1:10080"]}] } ] } ] } } }, "http": { "servers": { "backend": { "listen": ["127.0.0.1:10080"], "routes": [ { "handle": [{ "handler": "static_response", "status_code": "200", "body": "Hello World\n", "headers": { "Content-Type": ["text/plain"] } }] } ] } } } } } ```Testing Go program
```go package main import ( "crypto/tls" "log" ) func main() { outerConn, err := tls.Dial("tcp", "127.0.0.1:10443", &tls.Config{ ServerName: "outer", InsecureSkipVerify: true, }) if err != nil { log.Fatal(err) } innerConn := tls.Client(outerConn, &tls.Config{ ServerName: "inner", InsecureSkipVerify: true, }) _, err = innerConn.Write([]byte("GET / HTTP/1.0\r\nHost: localhost\r\n\r\n")) if err != nil { log.Fatal(err) } buf := make([]byte, 1024) n, err := innerConn.Read(buf) if err != nil { log.Fatal(err) } println(string(buf[:n])) } ```Should contain "Hello World" in the output.
endless loop bug
This is a reproducer for a bug that was introduced in #196. It was triggered by a non terminal handler that did not consume its bytes and would thus be matched again in the next matching loop iteration. Normally a call to prefetch should have happened after the first loop iteration, which blocks the endless loop until timeout. In this example config a proxy_protocol handler is used that does not accept the proxy protocol send by the test command and thus the matched bytes are not consumed.
References:
endless-loop-bug.json
```json { "admin": { "disabled": true }, "logging": { "logs": { "default": {"level":"DEBUG", "encoder": {"format":"console"}} } }, "apps": { "layer4": { "servers": { "loop": { "listen": ["0.0.0.0:8080"], "routes": [ { "match": [ {"proxy_protocol": {}} ], "handle": [ { "handler": "proxy_protocol", "allow": ["1.1.1.1/32"] } ] } ] } } } } } ```Testing commands:
Should fail after the
matching_timeout
(by default 3s) with a "aborted matching according to timeout" error in the caddy log. If the bug is triggered a caddy thread will run at 100% percent cpu load indefinitely and no error or connection stats are visible in the caddy log.listener wrapper fallback
This tests if it is possible to use the layer4 listener wrapper with caddy as fallback. Meaning connections not matched by layer4 are passed back to the caddy http app. This is a reproducer for a bug that was introduced with #192, that was fixed by #208.
References:
listener-wrapper-fallback.json
```json { "admin": { "disabled": true }, "logging": { "logs": { "default": {"level":"DEBUG", "encoder": {"format":"console"}} } }, "apps": { "tls": { "certificates": { "automate": ["localhost"] }, "automation": { "policies": [{ "subjects": ["localhost"], "issuers": [{ "module": "internal" }] }] } }, "http": { "servers": { "testing": { "listen": ["127.0.0.1:8080"], "listener_wrappers": [ { "wrapper": "layer4", "routes": [ { "match": [ { "tls": { "sni": ["localhost"] } } ], "handle": [ {"handler": "tls"}, { "handler": "proxy", "upstreams": [{"dial": ["127.0.0.1:10080"]}] } ] } ] } ], "routes": [ { "handle": [{ "handler": "static_response", "status_code": "200", "body": "Hello World from testing caddy\n", "headers": { "Content-Type": ["text/plain"] } }] } ] }, "backend": { "protocols": ["h1","h2","h2c"], "listen": ["127.0.0.1:10080"], "routes": [ { "handle": [{ "handler": "static_response", "status_code": "200", "body": "Hello World from backend caddy\n", "headers": { "Content-Type": ["text/plain"] } }] } ] } } } } } ```Testing commands:
The first (http) request should contain "Hello World from testing caddy" in the response. The second one (https) request should contain "Hello World from backend caddy" in the response.
listener wrapper matcher with no handler
This tests if it is possible to use a route with a matcher but without any handlers to workaround #207 in most cases.
References:
listener-wrapper-matcher-with-no-handler.json
```json { "admin": { "disabled": true }, "logging": { "logs": { "default": {"level":"DEBUG", "encoder": {"format":"console"}} } }, "apps": { "tls": { "certificates": { "automate": ["localhost"] }, "automation": { "policies": [{ "subjects": ["localhost"], "issuers": [{ "module": "internal" }] }] } }, "http": { "servers": { "testing": { "listen": ["127.0.0.1:8080"], "listener_wrappers": [ { "wrapper": "layer4", "routes": [ { "match": [ { "tls": { "sni": ["localhost"] } } ], "handle": [ {"handler": "tls"}, { "handler": "proxy", "upstreams": [{"dial": ["127.0.0.1:10080"]}] } ] }, { "match": [ {"http": []} ] } ] } ], "routes": [ { "handle": [{ "handler": "static_response", "status_code": "200", "body": "Hello World from testing caddy\n", "headers": { "Content-Type": ["text/plain"] } }] } ] }, "backend": { "protocols": ["h1","h2","h2c"], "listen": ["127.0.0.1:10080"], "routes": [ { "handle": [{ "handler": "static_response", "status_code": "200", "body": "Hello World from backend caddy\n", "headers": { "Content-Type": ["text/plain"] } }] } ] } } } } } ```Testing commands:
The first (http) request should contain "Hello World from testing caddy" in the response. The second one (https) request should contain "Hello World from backend caddy" in the response. At the time of writing this works.
matcher without io
This tests if it is possible to use a route with a matcher that does not read from the connection.
matcher-without-io.json
```json { "admin": { "disabled": true }, "logging": { "logs": { "default": {"level":"DEBUG", "encoder": {"format":"console"}} } }, "apps": { "layer4": { "servers": { "https": { "listen": ["0.0.0.0:10443"], "routes": [ { "match": [ {"remote_ip": {"ranges": ["0.0.0.0/0"]}} ], "handle": [ { "handler": "proxy", "upstreams": [{"dial": ["127.0.0.1:10080"]}] } ] } ] } } }, "http": { "servers": { "backend": { "protocols": ["h1","h2","h2c"], "listen": ["127.0.0.1:10080"], "routes": [ { "handle": [{ "handler": "static_response", "status_code": "200", "body": "Hello World\n", "headers": { "Content-Type": ["text/plain"] } }] } ] } } } } } ```Testing commands:
The first (ipv4) request should contain "Hello World" in the response. The second one (ipv6) request should fail with layer4 closing the connection.
server speaks first
This tests if it is possible to use layer4 as tcp proxy with protocols where the server speaks first. For example like SMTP or MariaDB.
References:
server-speaks-first.json
```json { "admin": { "disabled": true }, "logging": { "logs": { "default": {"level":"DEBUG", "encoder": {"format":"console"}} } }, "apps": { "layer4": { "servers": { "https": { "listen": ["0.0.0.0:10443"], "routes": [ { "handle": [ { "handler": "proxy", "upstreams": [{"dial": ["127.0.0.1:10080"]}] } ] } ] } } } } } ```Testing commands: Simulating a server that sends data after connection establishment.
Connecting to caddy as client
You should see "Hello World" being printed on the client shell. You have to restart the server nc command each time.
route without matchers after matcher
This tests if routes without matchers are not preferred when there exist matching routes before them.
This is a reproducer for a regression introduced in #208, that was fixed by #239.
References:
route-without-matchers-after-matcher.json
```json { "admin": { "disabled": true }, "logging": { "logs": { "default": {"level":"DEBUG", "encoder": {"format":"console"}} } }, "apps": { "layer4": { "servers": { "https": { "listen": ["0.0.0.0:10443"], "routes": [ { "match": [ {"http": [{"host": ["localhost"]}]} ], "handle": [ { "handler": "proxy", "upstreams": [{"dial": ["127.0.0.1:10080"]}] } ] }, { "handle": [ { "handler": "proxy", "upstreams": [{"dial": ["127.0.0.1:10081"]}] } ] } ] } } }, "http": { "servers": { "backend": { "protocols": ["h1"], "listen": ["127.0.0.1:10080"], "routes": [ { "handle": [{ "handler": "static_response", "status_code": "200", "body": "Hello from backend\n", "headers": { "Content-Type": ["text/plain"] } }] } ] }, "fallback": { "protocols": ["h1"], "listen": ["127.0.0.1:10081"], "routes": [ { "handle": [{ "handler": "static_response", "status_code": "200", "body": "Hello from fallback\n", "headers": { "Content-Type": ["text/plain"] } }] } ] } } } } } ```Testing commands:
The first curl should print "Hello from backend" and the second "Hello from fallback". During the regression both returned "Hello from fallback".
double matching not allowed
This tests if it is possible to match the same route multiple times. For example by recursively using tls in tls. This should not be possible, because it is confusing and often not what users want. But it was possible on some commits between #192 and #208.
References:
double-matching-not-allowed.json
```json { "admin": { "disabled": true }, "logging": { "logs": { "default": {"level":"DEBUG", "encoder": {"format":"console"}} } }, "apps": { "tls": { "certificates": { "automate": ["localhost"] }, "automation": { "policies": [{ "subjects": ["localhost"], "issuers": [{ "module": "internal" }] }] } }, "layer4": { "servers": { "https": { "listen": ["0.0.0.0:10443"], "routes": [ { "match": [ {"tls": {"sni": ["localhost"]}} ], "handle": [ {"handler": "tls"} ] }, { "match": [ {"http": [{"host": ["localhost"]}]} ], "handle": [ { "handler": "proxy", "upstreams": [{"dial": ["127.0.0.1:10080"]}] } ] } ] } } }, "http": { "servers": { "backend": { "protocols": ["h1","h2","h2c"], "listen": ["127.0.0.1:10080"], "routes": [ { "handle": [{ "handler": "static_response", "status_code": "200", "body": "Hello World\n", "headers": { "Content-Type": ["text/plain"] } }] } ] } } } } } ```Testing Go program
```go package main import ( "crypto/tls" "log" ) func main() { outerConn, err := tls.Dial("tcp", "127.0.0.1:10443", &tls.Config{ ServerName: "localhost", InsecureSkipVerify: true, }) if err != nil { log.Fatal(err) } innerConn := tls.Client(outerConn, &tls.Config{ ServerName: "localhost", InsecureSkipVerify: true, }) _, err = innerConn.Write([]byte("GET / HTTP/1.0\r\nHost: localhost\r\n\r\n")) if err != nil { log.Fatal(err) } buf := make([]byte, 1024) n, err := innerConn.Read(buf) if err != nil { log.Fatal(err) } println(string(buf[:n])) } ```The test program should print an EOF error since layer4 should immediately force close the connection after the first tls layer is processed. But on 6a8be7c4b8acb0c531b6151c94a9cd80894acce1 for example it would print "Hello World".
out of order matching not allowed
This tests if out of order route matching is possible. It should not be possible, because it is confusing and often not what users want. Instead it should only be possible to execute routes in the order they are defined in the config. For example this config is using a proxy protocol handler as second route after tls.
out-of-order-matching-not-allowed.json
```json { "admin": { "disabled": true }, "logging": { "logs": { "default": {"level":"DEBUG", "encoder": {"format":"console"}} } }, "apps": { "tls": { "certificates": { "automate": ["localhost"] }, "automation": { "policies": [{ "subjects": ["localhost"], "issuers": [{ "module": "internal" }] }] } }, "layer4": { "servers": { "https": { "listen": ["0.0.0.0:10443"], "routes": [ { "match": [ {"tls": {"sni": ["localhost"]}} ], "handle": [ {"handler": "tls"} ] }, { "match": [ {"proxy_protocol": {}} ], "handle": [ { "handler": "proxy_protocol", "allow": ["0.0.0.0/0"] } ] }, { "match": [ {"http": [{"host": ["localhost"]}]} ], "handle": [ { "handler": "proxy", "upstreams": [{"dial": ["127.0.0.1:10080"]}] } ] } ] } } }, "http": { "servers": { "backend": { "protocols": ["h1","h2","h2c"], "listen": ["127.0.0.1:10080"], "routes": [ { "handle": [{ "handler": "static_response", "status_code": "200", "body": "Hello World\n", "headers": { "Content-Type": ["text/plain"] } }] } ] } } } } } ```Testing commands:
This should immediately fail with layer4 closing the connection, because the proxy protocol route is not the first one. Or in other words the tls route should be unmatchable after the proxy protocol route was executed.