Bug report-Data Lake network rules can't use both UK South and UK West at the same time

ml-justinhewitt commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues

Community Note

Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Version of the module you are using

v0.0.5

Rover Version

No response

Terraform Version

No response

AzureRM Provider Version

No response

Affected Resource(s)/Data Source(s)

stsga3tmiareporting

Terraform Configuration Files

network = {
      bypass = ["AzureServices"] #optional. Valid options are any combination of Logging, Metrics, AzureServices, or None
      ip_rules = [
        "20.49.168.39",
        "217.22.14.250",
        "217.22.14.10",
        "4.158.0.0/15", # Following IP Addresses are AzureCloud.uksouth
        "4.234.0.0/16",
        "4.250.0.0/16",
        "13.87.64.0/19",
        "13.87.96.0/20",
        "13.104.129.128/26",
        "13.104.145.160/27",
        "13.104.146.64/26",
        "13.104.159.0/25",
        "20.0.0.0/16",
        "20.26.0.0/16",
        "20.38.106.0/23",
        "20.39.208.0/20",
        "20.39.224.0/21",
        "20.47.11.0/24",
        "20.47.34.0/24",
        "20.47.68.0/24",
        "20.47.114.0/24",
        "20.49.128.0/17",
        "20.50.96.0/19",
        "20.58.0.0/18",
        "20.60.17.0/24",
        "20.60.166.0/23",
        "20.68.0.0/18",
        "20.68.128.0/17",
        "20.77.0.0/17",
        "20.77.128.0/18",
        "20.90.64.0/18",
        "20.90.128.0/17",
        "20.95.67.0/24",
        "20.95.71.0/24",
        "20.95.74.0/23",
        "20.95.82.0/23",
        "20.95.84.0/24",
        "20.108.0.0/16",
        "20.117.64.0/18",
        "20.117.128.0/17",
        "20.135.176.0/22",
        "20.135.180.0/23",
        "20.150.18.0/25",
        "20.150.40.0/25",
        "20.150.41.0/24",
        "20.150.69.0/24",
        "20.157.28.0/24",
        "20.157.107.0/24",
        "20.157.112.0/24",
        "20.157.120.0/24",
        "20.157.157.0/24",
        "20.157.182.0/24",
        "20.157.246.0/24",
        "20.162.128.0/17",
        "20.190.143.0/25",
        "20.190.169.0/24",
        "20.202.4.0/24",
        "20.209.6.0/23",
        "20.209.30.0/23",
        "20.209.88.0/23",
        "20.209.128.0/23",
        "20.254.0.0/17",
        "40.64.144.200/29",
        "40.64.145.16/28",
        "40.79.215.0/24",
        "40.80.0.0/22",
        "40.81.128.0/19",
        "40.90.17.32/27",
        "40.90.17.160/27",
        "40.90.29.192/26",
        "40.90.128.112/28",
        "40.90.128.160/28",
        "40.90.131.64/27",
        "40.90.139.64/27",
        "40.90.141.192/26",
        "40.90.153.64/27",
        "40.90.154.0/26",
        "40.93.67.0/24",
        "40.101.57.192/26",
        "40.101.58.0/25",
        "40.120.32.0/19",
        "40.120.136.0/22",
        "40.126.15.0/25",
        "40.126.41.0/24",
        "51.11.0.0/18",
        "51.11.128.0/18",
        "51.104.0.0/19",
        "51.104.192.0/18",
        "51.105.0.0/18",
        "51.105.64.0/20",
        "51.132.0.0/18",
        "51.132.128.0/17",
        "51.140.0.0/17",
        "51.140.128.0/18",
        "51.141.128.32/27",
        "51.141.129.64/26",
        "51.141.130.0/25",
        "51.141.135.0/24",
        "51.141.192.0/18",
        "51.142.64.0/18",
        "51.142.192.0/18",
        "51.143.128.0/18",
        "51.143.208.0/20",
        "51.143.224.0/19",
        "51.145.0.0/17",
        "52.101.88.0/23",
        "52.101.95.0/24",
        "52.101.96.0/23",
        "52.102.164.0/24",
        "52.103.37.0/24",
        "52.103.165.0/24",
        "52.108.50.0/23",
        "52.108.88.0/24",
        "52.108.99.0/24",
        "52.108.100.0/23",
        "52.109.28.0/22",
        "52.111.242.0/24",
        "52.112.231.0/24",
        "52.112.240.0/20",
        "52.113.128.0/24",
        "52.113.200.0/22",
        "52.113.204.0/24",
        "52.113.224.0/19",
        "52.114.88.0/22",
        "52.120.160.0/19",
        "52.120.240.0/20",
        "52.123.141.0/24",
        "52.123.142.0/23",
        "52.136.21.0/24",
        "52.151.64.0/18",
        "52.239.187.0/25",
        "52.239.231.0/24",
        "52.245.64.0/22",
        "52.253.162.0/23",
        "104.44.89.224/27",
        "172.165.0.0/16",
        "172.166.0.0/15",
        "172.187.128.0/17",
      ] #optional. The prefix of ip_rules must be between 0 and 30 and only supports public IP addresses.
    }

Expected Behaviour

All the ip_rules for both uksouth and ukwest should be added

Actual Behaviour

If both uksouth and ukwest ip ranges are added the pipeline fails with error similar to:

Error: "network_rules.0.ip_rules.56" must start with IPV4 address and/or slash, number of bits (0-30) as prefix. Example: 23.45.1.0/30. │ │ with module.caf.module.storage_accounts["datalake1"].azurerm_storage_account.stg, │ on .terraform/modules/caf/modules/storage_account/storage_account.tf line 20, in resource "azurerm_storage_account" "stg": │ 20: resource "azurerm_storage_account" "stg" { │ ╵

Steps to Reproduce

No response

Important Factoids

No response

References

No response

ml-justinhewitt commented 1 year ago

I have found out why the data lake storage account IP rules fail when I try to use both UK South and UK West as firewall IP ranges. It's because storage accounts have a limit of 200 firewall IP rules and UK South and UK West combined have over 200 IP ranges:

https://learn.microsoft.com/en-us/azure/storage/common/storage-network-security?tabs=azure-portal#grant-access-from-an-internet-ip-range

ml-justinhewitt commented 1 year ago

AzureCloud.uksouth and AzureCloud.ukwest merged using https://iptoolsonline.net/ to produce a list with fewer than 200 entries

OuterSrc / terraform-azurerm-caf