allow_domain call frequency

[jasonbouffard]

We've implemented a web service to do domain validation. What we're curious about is the performance impact of issuing a web request in the allow_domain method? Also how often is this method called? Is it called ONLY just before the Let's Encrypt sequence?

It would seem if there is a cert for a domain in storage that we wouldn't need to call allow_domain.

Perhaps a bit more of this type of explanation would be useful in the docs.

Thank you.

[brianlund]

It is called whether you have a certificate or not. Personally I check if we already have a certificate before I ask our backend if it's ok to generate a new one.

-- first check if we have the certificate already, else check with backend if we allow it 
    local certstorage = auto_ssl:get("storage")

    local fullchain_pem, privkey_pem = certstorage:get_cert(domain)
    if fullchain_pem then
      return true
    else ...

I'm a bit concerned about overhead/performance impact also, but am still in planning stages and haven't gotten so far yet.

[jasonbouffard]

@brianlund thanks for sharing. I've implemented a local lua_shared_dict to 'cache' our results from said web service.

    auto_ssl:set("allow_domain", function(domain)
      local http = require "resty.http"
      local httpc = http.new()
      local cjson = require "cjson"
      local our_allow_domain_cache = ngx.shared.our_allow_domain_cache

      local found = our_allow_domain_cache:get(domain)

      if found == nil then
        local res, err = httpc:request_uri("[is domain allowed web service]"..domain, {
          method = "GET",
        })

        if not res then
          found = false
        else
          local data = cjson.decode(res.body)
          found = data.found
        end

        -- Cache for 60 seconds, for now...
        our_allow_domain_cache:set(domain, found, 60)
      else
        if found then
          -- Known good domain, keep it in cache for a little longer
          our_allow_domain_cache:set(domain, found, 300)
        end
      end

I really like your idea of checking the auto_ssl:get("storage") service. I might go with something similar. Can we assume this happens on every request? Or is lua_shared_dict auto_ssl getting used a caching layer for storage? It looks like it might be: https://github.com/GUI/lua-resty-auto-ssl/blob/master/lib/resty/auto-ssl/ssl_certificate.lua#L20

I wonder if it would be better in your example to check: ngx.shared.auto_ssl:get("domain:fullchain_der:" .. domain) vs calling out to storage directly.

[brianlund]

I believe ngx.shared.auto_ssl:get("domain:fullchain_der:" .. domain) only checks the shared memory cache. I plan to run multiple nodes so need something that is shared across them (the redis storage). I like your cache solution though, it would save a lot of calls to the storage service, if I see any problems with calling it that frequently, I'll keep that in mind.

[marcfowler]

I wanted to thank you @jasonbouffard for your demo code there - that was an extremely useful building block for me! Appreciate it!

[Alir3z4]

Can we assume this happens on every request?

This is a really good question.

My assumption is this method gets called when there's not certificate for the domain, but I'm not sure about it. Have you been able to confirm it @jasonbouffard ?

[max-starikevich]

@Alir3z4

I was curious about this as well. The docs aren't so transparent about this, so I had to inspect the source code.

So, on every request, auto-ssl does this:

• tries to get certs from ngx.shared.auto_ssl dictionary, which are cached for 1 hour
• calls a user-defined allow_domain method
• tries to get certs from storage
• creates a new certificate

Would be perfect to explain this in README.md. Thanks!

The source:

https://github.com/auto-ssl/lua-resty-auto-ssl/blob/86d3c94807ad1585b464fad2d4defb379f9c152a/lib/resty/auto-ssl/ssl_certificate.lua#L104-L157