clusterinthecloud / terraform

Terraform config for Cluster in the Cloud
https://cluster-in-the-cloud.readthedocs.io
MIT License
20 stars 23 forks source link

Validate VCN Security List Rules for File Storage #30

Closed christopheredsall closed 4 years ago

christopheredsall commented 5 years ago

Oracle notifies us:

The Oracle Cloud Infrastructure team has identified an issue where customers without stateful VCN security list egress rules may experience a temporary disruption during Mount Target failover. To function correctly, File Storage requires stateful ingress to TCP ports 111, 2048, 2049, and 2050 and stateful ingress to UDP ports 111 and 2048. File storage also requires stateful egress from TCP ports 111, 2048, 2049, and 2050 and stateful egress from UDP port 111. If you do not have the correctly configured stateful egress rules your Mount Target(s) can become temporarily unavailable during a planned or unplanned Mount Target failover. Please validate your security list rules as soon as possible. Instructions can be found in the additional information section of this message. Review the Configuring VCN Security List Rules for File Storage documentation found here: https://docs.cloud.oracle.com/iaas/Content/File/Tasks/securitylistsfilestorage.htm

christopheredsall commented 5 years ago

Default Security List

The default security list does not allow NFS traffic:

Stateless Source IP Protocol Source Port Range Destination Port Range Type and Code Allows
No 0.0.0.0/0 TCP All 22 TCP traffic for ports: 22 SSH Remote Login Protocol
No 0.0.0.0/0 ICMP 3, 4 ICMP traffic for: 3, 4 Destination Unreachable: Fragmentation Needed and Don't Fragment was Set
No 10.1.0.0/16 ICMP 3 ICMP traffic for: 3 Destination Unreachable
[usre@host ~]$ oci network security-list get --security-list-id=${SEC_LIST_ID} 
{
  "data": {
    "compartment-id": "ocid1.compartment.oc1..aaaaaaaap76l4pmddjqc4xs3foytrlxjaemhsteenmvxgcile43nmjtmm44q", 
    "defined-tags": {}, 
    "display-name": "Default Security List for ClusterVCN", 
    "egress-security-rules": [
      {
        "destination": "0.0.0.0/0", 
        "destination-type": "CIDR_BLOCK", 
        "icmp-options": null, 
        "is-stateless": false, 
        "protocol": "all", 
        "tcp-options": null, 
        "udp-options": null
      }
    ], 
    "freeform-tags": {}, 
    "id": "ocid1.securitylist.oc1.iad.aaaaaaaaapgg7m7nkb5dnx7n6ohj5ztjc2k6adjwqetrlknfdsvheoz3eklq", 
    "ingress-security-rules": [
      {
        "icmp-options": null, 
        "is-stateless": false, 
        "protocol": "6", 
        "source": "0.0.0.0/0", 
        "source-type": "CIDR_BLOCK", 
        "tcp-options": {
          "destination-port-range": {
            "max": 22, 
            "min": 22
          }, 
          "source-port-range": null
        }, 
        "udp-options": null
      }, 
      {
        "icmp-options": {
          "code": 4, 
          "type": 3
        }, 
        "is-stateless": false, 
        "protocol": "1", 
        "source": "0.0.0.0/0", 
        "source-type": "CIDR_BLOCK", 
        "tcp-options": null, 
        "udp-options": null
      }, 
      {
        "icmp-options": {
          "code": null, 
          "type": 3
        }, 
        "is-stateless": false, 
        "protocol": "1", 
        "source": "10.1.0.0/16", 
        "source-type": "CIDR_BLOCK", 
        "tcp-options": null, 
        "udp-options": null
      }
    ], 
    "lifecycle-state": "AVAILABLE", 
    "time-created": "2019-06-18T10:59:38.045000+00:00", 
    "vcn-id": "ocid1.vcn.oc1.iad.aaaaaaaaxzh7u7tonlrvp4th7wh743auhvvyxmbeuejyryloqrk4vr7doija"
  }, 
  "etag": "51ce8358"
}

CitC Security List

The firewall for the cluster network is set in https://github.com/ACRC/oci-cluster-terraform/blob/e8dcbb41d2f2d47f7ede8d12322acf41206840c1/network.tf#L36-L59

And this defines an ingress security list

Stateless Source IP Protocol Source Port Range Destination Port Range Type and Code Allows
No 10.0.0.0/8 TCP All All TCP traffic for ports: All
No 0.0.0.0/0 TCP All 3000 TCP traffic for ports: 3000
[user@host ~]$ oci network security-list get --security-list-id=${SEC_LIST_ID} 
{
  "data": {
    "compartment-id": "ocid1.compartment.oc1..aaaaaaaap76l4pmddjqc4xs3foytrlxjaemhsteenmvxgcile43nmjtmm44q", 
    "defined-tags": {}, 
    "display-name": "ClusterSecurityList", 
    "egress-security-rules": [], 
    "freeform-tags": {}, 
    "id": "ocid1.securitylist.oc1.iad.aaaaaaaalj4an5ghttm4gohw6yyogq2pfyukcr6g6n2mxp4oixptzhovxkbq", 
    "ingress-security-rules": [
      {
        "icmp-options": null, 
        "is-stateless": false, 
        "protocol": "6", 
        "source": "10.0.0.0/8", 
        "source-type": "CIDR_BLOCK", 
        "tcp-options": null, 
        "udp-options": null
      }, 
      {
        "icmp-options": null, 
        "is-stateless": false, 
        "protocol": "6", 
        "source": "0.0.0.0/0", 
        "source-type": "CIDR_BLOCK", 
        "tcp-options": {
          "destination-port-range": {
            "max": 3000, 
            "min": 3000
          }, 
          "source-port-range": null
        }, 
        "udp-options": null
      }
    ], 
    "lifecycle-state": "AVAILABLE", 
    "time-created": "2019-06-18T10:59:38.725000+00:00", 
    "vcn-id": "ocid1.vcn.oc1.iad.aaaaaaaaxzh7u7tonlrvp4th7wh743auhvvyxmbeuejyryloqrk4vr7doija"
  }, 
  "etag": "ac4e45d8"
}

There are no egress security lists.

Verify

As I read it the required ports are covered by the broad list in ClusterSecurityList. I don't thing we need the egress security list as we are not mounting the NFS mount target from outside the cluster VCN.