Add the ability to update Cloudflare WAF lists with detected IP addresses

[WolfRamAlpha12]

Unsure if this falls within the scope of this project, but I have a feature request:

• Whenever the IP updates, in addition to updating the DNS record, add or update an IP access rule (Security>WAF>Tools) to allow the new IP.

Here are the relevant docs: https://developers.cloudflare.com/api/operations/ip-access-rules-for-a-zone-list-ip-access-rules. It seems CF does not allow the value to be edited (only mode and notes), so upon an IP change the old one would have to be deleted and a new rule with the new IP would be created. For deletion, the API supports filtering the GET results using several optional parameters, including the notes.

Let me know if I can help out in implementing this is any way.

[favonia]

Thanks for the suggestion. I have to admit I'm pretty ignorant about this part of Cloudflare API, and it does seem to be quite tricky. My immediate question is, is there a reliable way to indicate which rule(s) to delete without the invariant we are assuming between the notes on the Cloudflare server and the updater configuration? What could be helpful is that, if you have a working automatic system set up, I would like to learn how it's working now.

PS: I also have to admit that I couldn't devote enough time in this project before summer 2024 to implement interesting new features; even #632 is taking me too long 😟

[WolfRamAlpha12]

My immediate question is, is there a reliable way to indicate which rule(s) to delete without the invariant we are assuming between the notes on the Cloudflare server and the updater configuration?

The notes parameter can be set to an arbitrary string (say This rule was created by cloudflare-ddns on $timestamp) when creating the rule. The work flow may look like:

1. GET request to CF with the filters object (Unsure how it is actually implemented; based on their example, a filter with the notes string set to This rule was created by cloudflare-ddns on should return any rules with the above example string; alternatively store and append a UUID to the notes string and filter by that, the docs show a similar filter as an example). A successful request will return matching rules along with their unique identifier.
2. DELETE request to CF, passing the unique identifier returned above (Only one rule should be returned ideally since we delete the old one every time it updates, but maybe store results returned matching id: "*" and send a DELETE request for each)
3. POST request to CF, creating a new rule, passing the new IP, setting it to allow, and of course, the notes for future filtering.

I don't have an automation setup yet, I found the need for it just before I opened this issue. I'll work on it and post the code here whenever I make it.

[favonia]

@WolfRamAlpha12 I still feel the idea to assign meaning to supposedly totally unstructured notes is not very reliable. But let's put that aside for the moment. Could you possibly explain why PATCH would not work? It seems I could update the IP of a target? Found the statement.

[favonia]

@WolfRamAlpha12 Never mind. I saw the strange documentation that described what you said.

[WolfRamAlpha12]

meaning to supposedly totally unstructured notes is not very reliable.

Appending and filtering a UUID would ensure no false positives. Storing the previously set IP and filtering by that is possible as well, so a query with configuration.target =ip configuration.value =$previousIP mode=whitelist match =all would return rules with an IP of $previousip set to allow

[favonia]

@WolfRamAlpha12 I prefer not to relying on stateful checks because they are fragile. Consider these cases:

1. Some people use external cron to manage the scheduling so the program immediately exits after the IPs are updated.
2. You might also want to restart or even recreate the updater container during upgrades and reboots.
3. A Cloudflare API call can fail for many reasons, including timeout and temporary network outage. When that happens, you cannot reliably know whether the rule has been successfully updated. (That is, you might see that the API call fails, but the change might have been made on the Cloudflare server.)
4. While this shouldn't happen, maybe the rules could be updated or changed by other programs or you manually.

The current stateless design is robust partly because it will eventually fix any inconsistency itself even if all of the above happen. That is, its eventual correctness does not depend on the correctness of the local state. This is the level of robustness I would like to maintain, and it's still a bit difficult for me to see how to achieve that using the current API for the IP filtering rules, but maybe you could help me see it.

[favonia]

@WolfRamAlpha12 Just to clarify, I'm still happy to find a good approach with you and I thank you for the GitHub issue ❤️

[WolfRamAlpha12]

@favonia Thank you, appreciate your willingness to work with me here. I'll keep the above scenarios in mind while trying to make my automation and see if I can come up with something that can meet the current level of robustness.

[jdvuyk]

Hi guys.

I believe this feature is also what I am looking for. Whilst this tool seems amazing at doing the DDNS bit, its hard to use the outcome in a meaningful way within the cloudflare eco-system. I think the answer is 'lists'. You can create a WAF rule based on the contents of a cloudflare list. So if this tool updates an arbitrary lists content, as well as the actual A record, you now have a very powerful tool to both update the record AND make it useable within cloudflares own WAF rules to do whatever you want.

Manage Account --> Configurations --> Lists https://developers.cloudflare.com/waf/tools/lists/custom-lists/

If it were me, I would have the tool create and manage its own list (like "cloudflare-ddns") and whenever the IP address gets change/updated, delete all items in the list and create one single item that represents the current IP. This is a 'reserved' list that can be used in WAF rules if the user wants to, or simply ignored if users don't need that.

[favonia]

@jdvuyk Thank you for providing the technical bits! I guess we can blame the users if they dare touch a list named ip-addresses-reported-by-cloudflare-ddns...? Now I can better see how these can be made to work!

Question: should we have one list for IPv4 and another for IPv6?

Also tagging @WolfRamAlpha12

PS: I'm still quite burdened by my current daytime job... until 2024 summer at least. I wish to conclude #632 first. But I can do these "easy" discussions. :grin:

[favonia]

Now the shoutrrr support is done, I'm back on track to implement this exciting feature :star_struck:

@jdvuyk I need your feedback about whether we should have separate lists. The free plan seems to allow only one list, so maybe I should put both the IPv4 and IPv6 addresses in the same list? Would that work well in your scenario?

Also, is IP_LISTS a good environment variable name to enable this feature. It will look like this:

IP_LISTS=the-id-of-the-list

[favonia]

@WolfRamAlpha12 I was checking the details of the Cloudflare Lists and I wonder if you are fine with its limitations on IPv6 addresses, that you can only block/allow a /64 range instead of an IP? It seems only individual rules can have single IPv6 addresses. On the other hand, Cloudflare suggests that blocking/allowing a single IPv6 IP makes less sense (and thus the limitation).

[WolfRamAlpha12]

I don't think I'll be using any IPv6 functionality, so I'm good either way. IPv6 adoption is fairly limited in my experience, don't think my ISP even assigns me an IPv6 address

[favonia]

Alright! I guess you don't care whether I put IPv4 and IPv6 addresses in the same list then 😆

[favonia]

@WolfRamAlpha12 When DELETE_ON_STOP=1, do you want the IP list to be deleted as well when the program exits? Actually, what would happen when a rule refers to a non-existing list? :thinking:

[WolfRamAlpha12]

In my head deleting the list would be expected behavior if DELETE_ON_STOP=1. Unsure what would happen, might be worth testing, or just warning users of potential issues if they use both together

[favonia]

I seem to be getting "This list is referenced by at least one ruleset rule (Code: 10041)", so Cloudflare has prevented dangling references. :-)

[favonia]

@WolfRamAlpha12 Another design question (sorry!): suppose your IP list has both IPv4 and IPv6 addresses/ranges, and you ask the updater to handle only IPv4. Do you want the updater to keep the existing IPv6 addresses/ranges intact? Or, should it remove IPv6 addresses/ranges when updating the list?

[favonia]

PS: I'm leaning towards keeping any addresses not "managed" by the updater, but I'm curious about what you think.

[favonia]

@WolfRamAlpha12 Yet another design question that is perhaps more important. I'm struggling with coming up with good environment variable names to configure this feature:

• WAF_LISTS: the comma-separated names of all the lists the updater should manage
• WAF_LIST_DESCRIPTION: the description for newly created lists
• WAF_LIST_COMMENT_SELECTOR: the planned, advanced feature to update only the IP addresses with a specific comment and leave other addresses alone

Any suggestions or opinions about the names? Is WAF clear to you? We will be stuck with whatever names we choose once the feature matures due to backward compatibility.

[favonia]

@WolfRamAlpha12 @jdvuyk Hi, I just want you to know that, the feature to update WAF lists has been implemented, and after implementing the Happy Eyeballs algorithm for something unrelated (see #883), 1.14.0 will be released. Once 1.14.0 released, I will be much more reluctant to change the interface due to backward compatibility. Let me know if you want more time to test it, or I will ship it very soon!

To test the development version, use the edge Docker tag. The README has been updated with this new feature.