placeholders not support {http.request.host}

[yulewang]

Hi,

I have a generic domain name and would like to use the replace-response to reach the replacement of the real domain name requested by the user. But always replace by text:{http.request.host}

As in the following configuration file:

*.example.com:443 {
  replace {
    example.com {http.request.host}
  }

Is there any way to make it happen? Thanks!

[francislavoie]

Please share your full config. Show an example request with curl -v. Show your logs.

[yulewang]

Actually, I have a site deployed to cloudflare. I'm hoping to use Caddy to enable a generic domain name address proxy the "cloudflare domain" for escape some blocking. And using replace-response to replace the body contents with the domain name of the user's real request.

Caddy --version:

v2.7.6

Caddyfile:

{
  order replace after encode
}

*.Bexample.com:443 {
  tls {
    dns cloudflare xxxxxxxxxx
  }
  log {
    output file /var/log/caddy/Bexample.com.log
  }
  encode gzip
  replace {
    Aexample.com {http.request.host} //Here's the part that won't work: the get is the text: {http.request.host}, not the user request "*.Bexample.com"
  }
  reverse_proxy * https://Aexample.com {
    header_up Host Aexample.com
    header_up Accept-Encoding identity

    header_up X-Real-IP {http.request.remote.host}
    header_up X-Forwarded-For {http.request.remote.host}
    header_up REMOTE-HOST {http.request.remote.host}
    header_down Set-Cookie Aexample.com Bexample.com
    header_up Referer {http.request.tls.server_name}
  }
}

[francislavoie]

We also need your logs (enable the debug global option) and an example request with curl -v.

It's important. Whether it works or not depends on how your upstream responds.

[yulewang]

sure, I created a test.html and content is 'Aexample.com' for show the "curl -v"

curl -v https://testsub.Bexample.com/clients/test.html

*   Trying ip:443...
* Connected to testsub.Bexample.com (ip) port 443 
(#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=*.Bexample.com
*  start date: Feb  5 08:53:39 2024 GMT
*  expire date: May  5 08:53:38 2024 GMT
*  subjectAltName: host "testsub.Bexample.com" matched cert's
 "*.Bexample.com"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after u
pgrade: len=0
* Using Stream ID: 1 (easy handle 0x559800709b20)
> GET /clients/test.html HTTP/2
> Host: testsub.Bexample.com
> user-agent: curl/7.74.0
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200 
< accept-ranges: bytes
< alt-svc: h3=":443"; ma=2592000
< cf-cache-status: DYNAMIC
< cf-ray: 852a5477fbf72abb-LAX
< content-type: text/html
< date: Fri, 09 Feb 2024 07:16:58 GMT
< last-modified: Fri, 09 Feb 2024 06:43:04 GMT
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/
report\/v3?s=z%2B6oC5GpNhlhCMLeZJPGZivKJUEM0NLPhddh%2FWpaoRNaY8TH9FOlz2
dtKqXhOj1SAiyjWS%hddhdh%2FnmN9wokIP2lhbwH4ai%2FpjSb8neNl
9jD%2Bl0lEAsHD5Zj3fjfjfJVRaG0Ck3mmSdDik%3D"}],"group":"cf-nel","max_
age":604800}
< server: Caddy
< server: cloudflare
< strict-transport-security: max-age=31536000
< 
* Connection #0 to host testsub.Bexample.com left intact
{http.request.host}

the last line is: {http.request.host}

[mholt]

Is your backend compressing the response (does it support gzip compression)? If so, the replacer can't operate on compressed payloads...

[eth-limo]

@mholt @francislavoie I was able to confirm this issue as well using a simple NodeJS backend for testing:

const http = require("http");

const server = http.createServer((req, res, next) => {
  console.log(req.url);
  console.log(req.rawHeaders);
  res.writeHead(200);
  res.write("This is a test", "utf8");
  res.end();
});

server.listen(process.argv[2]);

$ node index.js 8181

Using the following Caddyfile:

{
    admin off
    auto_https off

    local_certs

    log {
        level DEBUG
        format console
    }

    order replace after encode
}

:8443 {
    log {
        level INFO
        format console
    }

    bind 0.0.0.0

    tls internal {
        on_demand
    }

    reverse_proxy http://localhost:8181

    replace {
        "test" "I was replaced"
    }
}

Responses are rewritten when using a string with the replacement handler: "I was replaced":

curl https://localhost:8443 -v
*   Trying 127.0.0.1:8443...
* Connected to localhost (127.0.0.1) port 8443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
*  CAfile: /etc/pki/tls/certs/ca-bundle.crt
*  CApath: none

< HTTP/2 200 
< alt-svc: h3=":8443"; ma=2592000
< date: Tue, 16 Apr 2024 19:08:36 GMT
< server: Caddy
< 

This is a I was replaced%                        

However when using placeholder {host} or {http.request.host} for the replacement value, {http.request.host} is returned instead of being properly interpolated.

Caddyfile:

{
    admin off
    auto_https off

    local_certs

    log {
        level DEBUG
        format console
    }

    order replace after encode
}

:8443 {
    log {
        level INFO
        format console
    }

    bind 0.0.0.0

    tls internal {
        on_demand
    }

    reverse_proxy http://localhost:8181

    replace {
        "test" {host}
    }
}

Response:

curl https://localhost:8443 -v        
*   Trying 127.0.0.1:8443...
* Connected to localhost (127.0.0.1) port 8443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
*  CAfile: /etc/pki/tls/certs/ca-bundle.crt
*  CApath: none

< HTTP/2 200 
< alt-svc: h3=":8443"; ma=2592000
< date: Tue, 16 Apr 2024 19:09:57 GMT
< server: Caddy
< 

This is a {http.request.host}%  

[mholt]

Oh, you know, this is because, for efficiency reasons, we allocate the transformer chain at Provision-time instead of Request-time. So there is no {http.request.host} at the time the transformer chain is created.

I don't know for sure, but @icholy's excellent replace package does have methods like RegexpStringFunc() which might(?) allow us to allocate the transformers once which end up calling our function at replace-time, thus allowing us to evaluate placeholders at request-time.

There is no StringFunc() method though. @icholy Am I on the right track, for enabling dynamic evaluation of what the replacement values should be?

[eth-limo]

Oh, you know, this is because, for efficiency reasons, we allocate the transformer chain at Provision-time instead of Request-time. So there is no {http.request.host} at the time the transformer chain is created.

I don't know for sure, but @icholy's excellent replace package does have methods like RegexpStringFunc() which might(?) allow us to allocate the transformers once which end up calling our function at replace-time, thus allowing us to evaluate placeholders at request-time.

There is no StringFunc() method though. @icholy Am I on the right track, for enabling dynamic evaluation of what the replacement values should be?

Great! Looking forward to seeing what can be done. In our use case, we don't always know what the replacement value should be since we support many vhosts, so being able to dynamically replace would be amazing.

[Jonathazn]

Hello,

I'm trying to replace ngcspnonce in an angular app with this module as well as https://github.com/luludotdev/caddy-requestid so that it's a different nonce on every request. In the header the caddy-requestid module is correctly setting the nonce, but when I try to replace it using this module, it's replacing the string with the string literal. Am I correct in assuming this is because of the same issue here?

My Caddyfile:

https://www.subdomain.domain.tld {
    reverse_proxy example:80

    # Define the request ID module
    request_id {
    }

    # Define the replace-response module
    replace {
        nonce {request_id} # nonce is the value for ngcspnonce in the angular app, nonce becomes {request_id}
    }
}