auto-ssl / lua-resty-auto-ssl

On the fly (and free) SSL registration and renewal inside OpenResty/nginx with Let's Encrypt.
MIT License
1.93k stars 182 forks source link

How To Clear Queued Renewal Work? #297

Open surgiie opened 2 months ago

surgiie commented 2 months ago

Problem Description:

We are encountering issues with queued renewal work in the lua-resty-auto-ssl plugin. Specifically, we are using the Redis storage driver with the following configuration:

auto_ssl:set("storage_adapter", "resty.auto-ssl.storage_adapters.redis")
auto_ssl:set("redis", { host = "..."})

Our setup has resulted in a significant number of renewals getting queued for processing that are no longer necessary. As a consequence, our web server's CPU usage spikes, leading to severe disruptions in user traffic.

Current Workaround:

As a temporary solution, we have disabled the plugin using the following Lua block:

ssl_certificate_by_lua_block {
    auto_ssl:ssl_certificate({ generate_certs = false })
}

While this workaround has mitigated the issue, we are seeking a more permanent solution to clear out the unnecessary queued renewal work which starts to process back up once the plugin is re-enabled.

Desired Solution:

We are unsure of where this plugin is storing data for the renewals but we suspect that clearing specific key(s) from the Redis storage would resolve this problem. However, we are uncertain about which key(s) are pertinent to this issue and how to clear them effectively.

Requested Assistance:

Could someone knowledgeable about lua-resty-auto-ssl and Redis storage provide guidance on the following:

Which specific key(s) in Redis correspond to the queued/renewal work that needs to be cleared? What is the recommended method or script to clear out this unnecessary work from Redis? Any insights or assistance would be greatly appreciated as we aim to optimize our system's performance and minimize disruptions to user traffic.

surgiie commented 2 months ago

We ended up digging into redis and purging the keys out manually. We were hoping for a method via the plugin that would of prevented the need to manually go into the redis storage but Im posting here in hopes that it helps someone out:

We wrote out the relevant keys to a file:

redis-cli -h redis-host-example.com --scan --pattern '*domain-example.com*'

Then deleted them manually:

for key in `cat /tmp/redis-keys.txt`; do
   redis-cli -h redis-host-example.com DEL $key
done