search

lsst-uk / somerville-operations

User issue reporting and tracking for the Somerville Cloud
0 stars 0 forks source link

Security groups no longer apply correctly. #112

Closed GregBlow closed 1 year ago

GregBlow commented 1 year ago

Same instance with the same security groups set, but before and after being toggled on and off:

before:

gblow@EPCC-WIN-P12:~$ nmap 192.41.122.112 -Pn
Starting Nmap 7.80 ( https://nmap.org/ ) at 2023-08-01 15:16 BST
Nmap scan report for 192.41.122.112
Host is up (0.036s latency).
Not shown: 999 filtered ports
PORT   STATE SERVICE
22/tcp open  ssh

Nmap done: 1 IP address (1 host up) scanned in 10.32 seconds

after:

gblow@EPCC-WIN-P12:~$ nmap 192.41.122.112 -Pn
Starting Nmap 7.80 ( https://nmap.org/ ) at 2023-08-01 15:28 BST
Nmap scan report for 192.41.122.112
Host is up (0.038s latency).
Not shown: 987 filtered ports
PORT     STATE  SERVICE
22/tcp   open   ssh
80/tcp   open   http
443/tcp  closed https
5000/tcp closed upnp
8000/tcp closed http-alt
8080/tcp closed http-proxy
8081/tcp closed blackice-icecap
8082/tcp closed blackice-alerts
8083/tcp closed us-srv
8084/tcp closed unknown
8443/tcp closed https-alt
8651/tcp closed unknown
9001/tcp closed tor-orport

Nmap done: 1 IP address (1 host up) scanned in 6.03 seconds
GregBlow commented 1 year ago

some superseding rule seems to be being applied to unfilter filtered ports (e.g. port 80 is now unfiltered in the above example)

GregBlow commented 1 year ago

logs from removing/adding security group to Qserv instance (tenant_id 9168... is Qserv)

2023-08-01 15:38:43.807 26 INFO neutron.wsgi [req-5c423d35-9224-4860-b069-758f43a9dcfe 3ad62106189e426f87d3233161e060ec 9168c636eaec419f807c46f1454e87a9 - default default] 10.19.3.1,10.19.3.10 "GET /v2.0/ports?device_id=4e33ce3f-b88d-4b21-add5-bc4d5ef7ac8e HTTP/1.1" status: 200  len: 1158 time: 0.0358651
2023-08-01 15:38:43.906 25 INFO neutron.wsgi [req-b0d248b8-740c-425e-8404-cf0b7921f106 3ad62106189e426f87d3233161e060ec 9168c636eaec419f807c46f1454e87a9 - default default] 10.19.3.200,10.19.3.10 "GET /v2.0/security-groups?tenant_id=9168c636eaec419f807c46f1454e87a9&shared=False HTTP/1.1" status: 200  len: 13158 time: 0.0665221
2023-08-01 15:38:44.387 25 INFO neutron.db.ovn_revision_numbers_db [req-d022ba5e-c333-4854-b31f-71f02972fd83 3ad62106189e426f87d3233161e060ec 9168c636eaec419f807c46f1454e87a9 - default default] Successfully bumped revision number for resource 08ea657a-5d07-4fed-8a85-8940333b8eb2 (type: ports) to 11
2023-08-01 15:38:44.399 25 INFO neutron.wsgi [req-d022ba5e-c333-4854-b31f-71f02972fd83 3ad62106189e426f87d3233161e060ec 9168c636eaec419f807c46f1454e87a9 - default default] 10.19.3.200,10.19.3.10 "PUT /v2.0/ports/08ea657a-5d07-4fed-8a85-8940333b8eb2 HTTP/1.1" status: 200  len: 1205 time: 0.3498216
2023-08-01 15:38:44.507 25 INFO neutron.notifiers.nova [-] Nova event matching ['req-9e3e16d9-970b-45dd-ac98-f3be33502f84'] response: {'name': 'network-changed', 'server_uuid': '4e33ce3f-b88d-4b21-add5-bc4d5ef7ac8e', 'tag': '08ea657a-5d07-4fed-8a85-8940333b8eb2', 'status': 'completed', 'code': 200}
2023-08-01 15:38:44.676 25 INFO neutron.wsgi [req-f9597337-c0d7-4c51-9e62-f12a09dbd7a3 3ad62106189e426f87d3233161e060ec 9168c636eaec419f807c46f1454e87a9 - default default] 10.19.3.15,10.19.3.10 "GET /v2.0/ports?device_id=4e33ce3f-b88d-4b21-add5-bc4d5ef7ac8e HTTP/1.1" status: 200  len: 1236 time: 0.0410380
2023-08-01 15:38:44.774 25 INFO neutron.wsgi [req-b64a9b87-ff1e-42d1-bd22-03a9eb8c626f aed07b48ef2e432180d94fc79a3ed730 0a1e36b0ecb547d49115eb79b36aea66 - default default] 10.19.3.59,10.19.3.10 "GET /v2.0/subnets?id=3a7687de-f4dc-4d47-8d2e-02099b3ceb92 HTTP/1.1" status: 200  len: 839 time: 0.0374620
2023-08-01 15:38:44.800 25 INFO neutron.wsgi [req-b8a906e0-c5f3-41ba-b525-a7749f7fc9ab aed07b48ef2e432180d94fc79a3ed730 0a1e36b0ecb547d49115eb79b36aea66 - default default] 10.19.3.59,10.19.3.10 "GET /v2.0/ports?network_id=3c7b83ed-b695-4d08-b8bf-7a3ef24a0cb7&device_owner=network%3Adhcp HTTP/1.1" status: 200  len: 186 time: 0.0219975
2023-08-01 15:38:44.869 25 INFO neutron.wsgi [req-b10cfd81-3bfc-4c13-ad65-6366316afb42 aed07b48ef2e432180d94fc79a3ed730 0a1e36b0ecb547d49115eb79b36aea66 - default default] 10.19.3.59,10.19.3.10 "GET /v2.0/networks/3c7b83ed-b695-4d08-b8bf-7a3ef24a0cb7?fields=segments HTTP/1.1" status: 200  len: 188 time: 0.0642142
2023-08-01 15:38:44.990 25 INFO neutron.wsgi [req-e3fa6c64-f2a1-4fc4-9a3e-b0aa1881f157 3ad62106189e426f87d3233161e060ec 9168c636eaec419f807c46f1454e87a9 - default default] 10.19.3.200,10.19.3.10 "GET /v2.0/networks?id=3c7b83ed-b695-4d08-b8bf-7a3ef24a0cb7 HTTP/1.1" status: 200  len: 824 time: 0.0710726
2023-08-01 15:38:47.577 23 INFO neutron.wsgi [req-36e51e07-4658-4b9d-8c2c-e38913f8d84c aed07b48ef2e432180d94fc79a3ed730 0a1e36b0ecb547d49115eb79b36aea66 - default default] 10.19.3.56,10.19.3.10 "GET /v2.0/ports?device_id=d99c453b-b12b-4ba9-bfd9-1c76bec54262&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200  len: 243 time: 0.0407722
2023-08-01 15:38:47.792 23 INFO neutron.wsgi [req-76f17b15-33f6-40d2-abe1-5f9776a1ed56 aed07b48ef2e432180d94fc79a3ed730 0a1e36b0ecb547d49115eb79b36aea66 - default default] 10.19.3.56,10.19.3.10 "GET /v2.0/floatingips?fixed_ip_address=10.65.0.85&port_id=cfe14b4f-7abf-4f99-99e2-c3915c99bbc5 HTTP/1.1" status: 200  len: 1013 time: 0.0451636
2023-08-01 15:38:47.869 23 INFO neutron.wsgi [req-95bd2f16-3233-4055-9961-d2b10cccf66a aed07b48ef2e432180d94fc79a3ed730 0a1e36b0ecb547d49115eb79b36aea66 - default default] 10.19.3.56,10.19.3.10 "GET /v2.0/extensions HTTP/1.1" status: 200  len: 10892 time: 0.0048041
2023-08-01 15:38:50.532 26 INFO neutron.wsgi [-] 10.19.3.1 "GET / HTTP/1.1" status: 200  len: 227 time: 0.0040653
2023-08-01 15:38:57.101 22 INFO neutron.wsgi [req-791b4426-0b55-4354-a42b-20bbb12d4105 0c80eb85c10c4893a1be255f5eaa6444 ff3e2de6a0b844d581bcd4335c18d2a4 - default default] 10.19.3.15,10.19.3.10 "GET /v2.0/floatingips HTTP/1.1" status: 200  len: 55936 time: 0.1555777
2023-08-01 15:38:57.607 22 INFO neutron.wsgi [req-34ff1262-26e2-4f45-858f-a76ccf1dfddb aed07b48ef2e432180d94fc79a3ed730 0a1e36b0ecb547d49115eb79b36aea66 - default default] 10.19.3.13,10.19.3.10 "GET /v2.0/ports?tenant_id=5b5102968e5347ad98676ea42b6519df&device_id=6581cb40-fd54-4fb3-b354-b0ac9b8fcf7a HTTP/1.1" status: 200  len: 2242 time: 0.0375197
2023-08-01 15:38:57.808 22 INFO neutron.wsgi [req-1291f646-3888-4d15-9a18-f88244bab545 aed07b48ef2e432180d94fc79a3ed730 0a1e36b0ecb547d49115eb79b36aea66 - default default] 10.19.3.13,10.19.3.10 "GET /v2.0/subnets?id=250b3916-b9a5-41f5-ba12-556937963259 HTTP/1.1" status: 200  len: 837 time: 0.0367196
2023-08-01 15:38:57.904 23 INFO neutron.wsgi [req-c581842b-226b-47b9-9011-97c37dca96e8 0c80eb85c10c4893a1be255f5eaa6444 ff3e2de6a0b844d581bcd4335c18d2a4 - default default] 10.19.3.15,10.19.3.10 "GET /v2.0/security-groups?tenant_id=ff3e2de6a0b844d581bcd4335c18d2a4 HTTP/1.1" status: 200  len: 2971 time: 0.0409899
2023-08-01 15:38:58.469 22 INFO neutron.wsgi [req-182198cc-92a7-447e-bf5a-542ccf281bab aed07b48ef2e432180d94fc79a3ed730 0a1e36b0ecb547d49115eb79b36aea66 - default default] 10.19.3.13,10.19.3.10 "GET /v2.0/networks/1445f6fe-1257-431b-bac3-2c255d68d5f5?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200  len: 253 time: 0.0669451
2023-08-01 15:38:58.542 22 INFO neutron.wsgi [req-3a059074-237c-471d-b826-e1d9a775a372 aed07b48ef2e432180d94fc79a3ed730 0a1e36b0ecb547d49115eb79b36aea66 - default default] 10.19.3.13,10.19.3.10 "GET /v2.0/subnets?id=d94f3b30-c4c4-431b-8e85-d6304df5c843 HTTP/1.1" status: 200  len: 842 time: 0.0362089
2023-08-01 15:38:58.641 22 INFO neutron.wsgi [req-b8bd768f-53ee-44bc-9461-9557f14efe7d aed07b48ef2e432180d94fc79a3ed730 0a1e36b0ecb547d49115eb79b36aea66 - default default] 10.19.3.13,10.19.3.10 "GET /v2.0/networks/12a61257-7a3d-49c4-b379-540b9e61b83e?fields=segments HTTP/1.1" status: 200  len: 188 time: 0.0633645
2023-08-01 15:38:59.267 23 INFO neutron.wsgi [req-8ec39d5e-ef57-4531-ad88-32b4be8d000d 0c80eb85c10c4893a1be255f5eaa6444 ff3e2de6a0b844d581bcd4335c18d2a4 - default default] 10.19.3.15,10.19.3.10 "GET /v2.0/agents HTTP/1.1" status: 200  len: 11831 time: 0.0859740
2023-08-01 15:38:59.481 23 INFO neutron.wsgi [req-73055987-2741-43fe-b242-a459b3c246f4 0c80eb85c10c4893a1be255f5eaa6444 ff3e2de6a0b844d581bcd4335c18d2a4 - default default] 10.19.3.15,10.19.3.10 "GET /v2.0/networks HTTP/1.1" status: 200  len: 13587 time: 0.1124082
2023-08-01 15:39:00.078 23 INFO neutron.wsgi [req-31c6d57c-d774-4b1f-b067-40e544b73c42 0c80eb85c10c4893a1be255f5eaa6444 ff3e2de6a0b844d581bcd4335c18d2a4 - default default] 10.19.3.15,10.19.3.10 "GET /v2.0/routers HTTP/1.1" status: 200  len: 7300 time: 0.1116006
GregBlow commented 1 year ago 
(openstack-config) [stack@sv-admin-0 openstack-config]$ openstack security group list --project Qserv
+--------------------------------------+---------------+----------------------------------------+----------------------------------+------+
| ID                                   | Name          | Description                            | Project                          | Tags |
+--------------------------------------+---------------+----------------------------------------+----------------------------------+------+
| 1ed86e6b-8c4e-4ca3-90ef-f872d1acc664 | default       | Default security group                 | 9168c636eaec419f807c46f1454e87a9 | []   |
| 413c123a-7e1d-417c-b2d5-09c36256cea8 | qserv-jump-sg | Security group for the qserv jump node | 9168c636eaec419f807c46f1454e87a9 | []   |
| a202dbd3-b215-429f-b1b2-9791b0f1db09 | qserv-kube-sg |                                        | 9168c636eaec419f807c46f1454e87a9 | []   |
| ea30233b-6b2e-4847-b8de-bea4f4551e6d | qserv-mysql   |                                        | 9168c636eaec419f807c46f1454e87a9 | []   |
+--------------------------------------+---------------+----------------------------------------+----------------------------------+------+
GregBlow commented 1 year ago 
(openstack-config) [stack@sv-admin-0 openstack-config]$ openstack server list --project Qserv
+--------------------------------------+-------------------------+---------+-----------------------------------+--------------+-------------------+
| ID                                   | Name                    | Status  | Networks                          | Image        | Flavor            |
+--------------------------------------+-------------------------+---------+-----------------------------------+--------------+-------------------+
| 83cfdaa8-4635-4060-b53b-dafb3f39d294 | sv-qserv-dev-jump       | ACTIVE  | qserv=10.71.0.42, 192.41.122.69   | ubuntu-jammy | qserv-jump-dev    |
| ce200818-00ef-4b70-b897-83987eee25a1 | sv-qserv-dev-utility-1  | ACTIVE  | qserv=10.71.0.235                 | ubuntu-jammy | qserv-utility-dev |
| 2a4037da-5d20-4cc8-844c-ccc6a7ac78e9 | sv-qserv-dev-worker-1   | ACTIVE  | qserv=10.71.0.20                  | ubuntu-jammy | qserv-worker-dev  |
| 30628b9a-0f58-4914-b60f-af95a072fe9c | sv-qserv-dev-worker-3   | ACTIVE  | qserv=10.71.0.223                 | ubuntu-jammy | qserv-worker-dev  |
| 410d7875-040f-463e-826b-4c7df92e3494 | sv-qserv-dev-czar       | ACTIVE  | qserv=10.71.0.218                 | ubuntu-jammy | qserv-czar-dev    |
| 89a990dc-4398-4986-b152-fe8d857e6ad6 | sv-qserv-dev-worker-2   | ACTIVE  | qserv=10.71.0.198                 | ubuntu-jammy | qserv-worker-dev  |
| 4911435d-59e4-4486-8f9a-26a3073a11a6 | sv-qserv-test-worker-2  | ACTIVE  | qserv=10.71.0.94                  | ubuntu-jammy | qserv-worker-dev  |
| 4e33ce3f-b88d-4b21-add5-bc4d5ef7ac8e | sv-qserv-test-jump      | ACTIVE  | qserv=10.71.0.205, 192.41.122.174 | ubuntu-jammy | qserv-jump-dev    |
GregBlow commented 1 year ago

all networks have port_security_enabled=false

GregBlow commented 1 year ago

creating a new network with neutron should set port_security_enabled to true by default.

(openstack-config) [stack@sv-admin-0 openstack-config]$ neutron net-create port_security_test
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
Created a new network:
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| availability_zone_hints   |                                      |
| availability_zones        |                                      |
| created_at                | 2023-08-02T09:28:59Z                 |
| description               |                                      |
| dns_domain                |                                      |
| id                        | 5089d5ad-8349-4d14-877c-53e43c724e59 |
| ipv4_address_scope        |                                      |
| ipv6_address_scope        |                                      |
| is_default                | False                                |
| mtu                       | 8942                                 |
| name                      | port_security_test                   |
| project_id                | ff3e2de6a0b844d581bcd4335c18d2a4     |
| provider:network_type     | geneve                               |
| provider:physical_network |                                      |
| provider:segmentation_id  | 1476                                 |
| revision_number           | 1                                    |
| router:external           | False                                |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tags                      |                                      |
| tenant_id                 | ff3e2de6a0b844d581bcd4335c18d2a4     |
| updated_at                | 2023-08-02T09:28:59Z                 |
+---------------------------+--------------------------------------+
GregBlow commented 1 year ago 
(openstack-config) [stack@sv-admin-0 openstack-config]$ openstack network show port_security_test
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | UP                                   |
| availability_zone_hints   |                                      |
| availability_zones        |                                      |
| created_at                | 2023-08-02T09:28:59Z                 |
| description               |                                      |
| dns_domain                |                                      |
| id                        | 5089d5ad-8349-4d14-877c-53e43c724e59 |
| ipv4_address_scope        | None                                 |
| ipv6_address_scope        | None                                 |
| is_default                | None                                 |
| is_vlan_transparent       | None                                 |
| mtu                       | 8942                                 |
| name                      | port_security_test                   |
| port_security_enabled     | False                                |
| project_id                | ff3e2de6a0b844d581bcd4335c18d2a4     |
| provider:network_type     | geneve                               |
| provider:physical_network | None                                 |
| provider:segmentation_id  | 1476                                 |
| qos_policy_id             | None                                 |
| revision_number           | 1                                    |
| router:external           | Internal                             |
| segments                  | None                                 |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tags                      |                                      |
| updated_at                | 2023-08-02T09:28:59Z                 |
+---------------------------+--------------------------------------+
GregBlow commented 1 year ago

all instances with floating IPs have been updated now. I cannot see any remaining holes by port scanning the public range.