Add standard for default security group rules

A quite new API (from 2023.2) allows the default security group rules to be edited instead of being hardcoded. This means, that the rules that are automatically added to each new security group (may it be a default or a custom group) can vary between CSPs and may result in security issues for customers, that expect a certain behavior for a default security group. E.g. only allowing outgoing traffic.

Example of adding new rules:

stack@devstack:~/devstack$ openstack default security group rule create --ingress --protocol tcp --dst-port 22 --for-default-sg --for-custom-sg
+-------------------------+--------------------------------------+
| Field                   | Value                                |
+-------------------------+--------------------------------------+
| description             |                                      |
| direction               | ingress                              |
| ether_type              | IPv4                                 |
| id                      | b0bfe643-7d17-4f85-96f3-fc6aab75748d |
| port_range_max          | 22                                   |
| port_range_min          | 22                                   |
| protocol                | tcp                                  |
| remote_address_group_id | None                                 |
| remote_group_id         | None                                 |
| remote_ip_prefix        | 0.0.0.0/0                            |
| used_in_default_sg      | True                                 |
| used_in_non_default_sg  | True                                 |
+-------------------------+--------------------------------------+
stack@devstack:~/devstack$ openstack default security group rule create --ingress --protocol tcp --dst-port 80 --description http
+-------------------------+--------------------------------------+
| Field                   | Value                                |
+-------------------------+--------------------------------------+
| description             | http                                 |
| direction               | ingress                              |
| ether_type              | IPv4                                 |
| id                      | 25391f6c-2521-4878-a046-e71c42040f2e |
| port_range_max          | 80                                   |
| port_range_min          | 80                                   |
| protocol                | tcp                                  |
| remote_address_group_id | None                                 |
| remote_group_id         | None                                 |
| remote_ip_prefix        | 0.0.0.0/0                            |
| used_in_default_sg      | False                                |
| used_in_non_default_sg  | True                                 |
+-------------------------+--------------------------------------+
stack@devstack:~/devstack$ openstack security group create new-test-group
+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field           | Value                                                                                                                                                                                    |
+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| created_at      | 2024-03-14T13:19:14Z                                                                                                                                                                     |
| description     | new-test-group                                                                                                                                                                           |
| id              | ae30070e-bcbd-4e5b-9ef2-27a569af2490                                                                                                                                                     |
| name            | new-test-group                                                                                                                                                                           |
| project_id      | 15f2ab0eaa5b4372b759bde609e86224                                                                                                                                                         |
| revision_number | 1                                                                                                                                                                                        |
| rules           | created_at='2024-03-14T13:19:14Z', direction='egress', ethertype='IPv6', id='25e5f5fb-3baa-4489-891d-e13c94af3b04', standard_attr_id='145', updated_at='2024-03-14T13:19:14Z'            |
|                 | created_at='2024-03-14T13:19:14Z', direction='ingress', ethertype='IPv4', id='26ae7373-8b67-46f0-a578-7421faf34430', normalized_cidr='0.0.0.0/0', port_range_max='22',                   |
|                 | port_range_min='22', protocol='tcp', remote_ip_prefix='0.0.0.0/0', standard_attr_id='147', updated_at='2024-03-14T13:19:14Z'                                                             |
|                 | created_at='2024-03-14T13:19:14Z', direction='ingress', ethertype='IPv4', id='5f1c8147-7a0f-4de3-b46e-43746f877933', normalized_cidr='0.0.0.0/0', port_range_max='80',                   |
|                 | port_range_min='80', protocol='tcp', remote_ip_prefix='0.0.0.0/0', standard_attr_id='144', updated_at='2024-03-14T13:19:14Z'                                                             |
|                 | created_at='2024-03-14T13:19:14Z', direction='egress', ethertype='IPv4', id='ad770fa2-682b-409b-9cbb-138dfb5e7b31', standard_attr_id='146', updated_at='2024-03-14T13:19:14Z'            |
| shared          | False                                                                                                                                                                                    |
| stateful        | True                                                                                                                                                                                     |
| tags            | []                                                                                                                                                                                       |
| updated_at      | 2024-03-14T13:19:14Z                                                                                                                                                                     |
+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

We should find a consensus about the default rules, that should be applied to SGs and write a corresponding standard.

This issue is also based on the Decision Record created here: https://github.com/SovereignCloudStack/standards/pull/495

Definition of Done:

Please refer to scs-0001-v1 for details.

[ ] Proposal has been written with name of the form scs-xxxx-v1-slug.md (only substitute slug)
[ ] Proposal has the fields status, type, track set
[ ] Proposal has been voted upon in the corresponding team
[ ] Status has been changed into Draft, file renamed: xxxx replaced by document number
[ ] If applicable: test script has been written (this item may be moved into a separate issue so long as the state is Draft)

Policy check:

stack@devstack:~/devstack$ openstack default security group rule list --fit-width
+---------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+--------------------------------+-------------------------------+
| ID                        | IP Protocol | Ethertype | IP Range  | Port Range | Direction | Remote Security Group | Remote Address Group | Used in default Security Group | Used in custom Security Group |
+---------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+--------------------------------+-------------------------------+
| 25391f6c-2521-4878-a046-  | tcp         | IPv4      | 0.0.0.0/0 | 80:80      | ingress   | None                  | None                 | False                          | True                          |
| e71c42040f2e              |             |           |           |            |           |                       |                      |                                |                               |
| 47b929fd-9b39-4f22-abc5-  | None        | IPv6      | ::/0      |            | egress    | None                  | None                 | True                           | True                          |
| 7d4f64d10909              |             |           |           |            |           |                       |                      |                                |                               |
| 6ace51bb-5258-45ab-9ba9-  | None        | IPv6      | ::/0      |            | ingress   | PARENT                | None                 | True                           | False                         |
| 1efbebfb086b              |             |           |           |            |           |                       |                      |                                |                               |
| 92a79600-5358-4fef-a390-  | None        | IPv4      | 0.0.0.0/0 |            | egress    | None                  | None                 | True                           | True                          |
| 822665f46070              |             |           |           |            |           |                       |                      |                                |                               |
| 997bb0c2-652e-4d1f-b910-  | None        | IPv4      | 0.0.0.0/0 |            | ingress   | PARENT                | None                 | True                           | False                         |
| e12c89f88b44              |             |           |           |            |           |                       |                      |                                |                               |
| b0bfe643-7d17-4f85-96f3-  | tcp         | IPv4      | 0.0.0.0/0 | 22:22      | ingress   | None                  | None                 | True                           | True                          |
| fc6aab75748d              |             |           |           |            |           |                       |                      |                                |                               |
+---------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+--------------------------------+-------------------------------+
stack@devstack:~/devstack$ openstack default security group rule create --ingress --protocol tcp --dst-port 2222 --description new-ssh
ForbiddenException: 403: Client Error for url: http://192.168.23.238:9696/networking/v2.0/default-security-group-rules, rule:create_default_security_group_rule is disallowed by policy

Listing all the default rules is allowed, but they cannot be edited by users with the role member.

One problem that occured is, that Security Groups work as whitlists only. So it is not possible, to e.g. allow all outgoing traffic and just disallow 1 or 2 ports to be used.

stack@devstack:~/devstack$ openstack security group rule create --egress --dst-port 1:24 --description "Default Egress Part 1" test-group
Error while executing command: BadRequestException: 400, Must also specify protocol if port range is given.

A better way would be to divide by protocol what is allowed: E.g. allow all outgoing TCP traffic.

We should just consider that the default rules MUST NOT get overwhelming. So too many rules will confuse users and they will not change or disable them, even though in their use case they can and should do this.

an interesting behavior is also, that when creating default rules, the rules for the default SG are also used for the custom SGs:

stack@devstack:~/devstack$ openstack default security group rule create --egress --ethertype IPv4 --for-custom-sg
+-------------------------+--------------------------------------+
| Field                   | Value                                |
+-------------------------+--------------------------------------+
| description             |                                      |
| direction               | egress                               |
| ether_type              | IPv4                                 |
| id                      | e7536f00-9866-4603-b193-beb9c3c0cd0f |
| port_range_max          | None                                 |
| port_range_min          | None                                 |
| protocol                | None                                 |
| remote_address_group_id | None                                 |
| remote_group_id         | None                                 |
| remote_ip_prefix        | 0.0.0.0/0                            |
| used_in_default_sg      | False                                |
| used_in_non_default_sg  | True                                 |
+-------------------------+--------------------------------------+
stack@devstack:~/devstack$ openstack default security group rule create --egress --ethertype IPv4 --for-default-sg
+-------------------------+--------------------------------------+
| Field                   | Value                                |
+-------------------------+--------------------------------------+
| description             |                                      |
| direction               | egress                               |
| ether_type              | IPv4                                 |
| id                      | d6d8b677-fead-45fe-9bfd-ffcb4c51acfd |
| port_range_max          | None                                 |
| port_range_min          | None                                 |
| protocol                | None                                 |
| remote_address_group_id | None                                 |
| remote_group_id         | None                                 |
| remote_ip_prefix        | 0.0.0.0/0                            |
| used_in_default_sg      | True                                 |
| used_in_non_default_sg  | True                                 |
+-------------------------+--------------------------------------+
stack@devstack:~/devstack$ openstack default security group rule list
+--------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+--------------------------------+-------------------------------+
| ID                       | IP Protocol | Ethertype | IP Range  | Port Range | Direction | Remote Security Group | Remote Address Group | Used in default Security Group | Used in custom Security Group |
+--------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+--------------------------------+-------------------------------+
| 47b929fd-9b39-4f22-abc5- | None        | IPv6      | ::/0      |            | egress    | None                  | None                 | True                           | True                          |
| 7d4f64d10909             |             |           |           |            |           |                       |                      |                                |                               |
| d6d8b677-fead-45fe-9bfd- | None        | IPv4      | 0.0.0.0/0 |            | egress    | None                  | None                 | True                           | True                          |
| ffcb4c51acfd             |             |           |           |            |           |                       |                      |                                |                               |
| e7536f00-9866-4603-b193- | None        | IPv4      | 0.0.0.0/0 |            | egress    | None                  | None                 | False                          | True                          |
| beb9c3c0cd0f             |             |           |           |            |           |                       |                      |                                |                               |
+--------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+--------------------------------+-------------------------------+

That means the parameters --for-default-sg and --for-custom-sg work differently.

But at least OpenStak checks for duplicates:

stack@devstack:~/devstack$ openstack default security group rule create --egress --ethertype IPv4 --for-custom-sg
ConflictException: 409: Client Error for url: http://192.168.23.238:9696/networking/v2.0/default-security-group-rules, Default Security group rule already exists. Rule id is 92a79600-5358-4fef-a390-822665f46070.

Transfering the discussion to this issue:

What should the default Security Rules look like?

There are different things described in the OpenStack documentation.

The most actual default is this: https://github.com/openstack/neutron/blob/master/neutron/db/migration/alembic_migrations/versions/2023.2/expand/c33da356b165_security_group_default_rules.py For default SGs:

all egress is allowed
ingress is only allowed from same SG

For custom SG:

all egress is allowed
no ingress is allowed

For stable/zed it is the same without the default security group endpoint: https://github.com/openstack/neutron/blob/stable/zed/neutron/db/securitygroups_db.py#L119

@garloff @fkr @artificial-intelligence @bitkeks @horazont @berendt a disussion with CSPs should help us define a good default here. Would it be possible to gather a lot of people in such a discussion?

Edit: I wonder, why by default the ingress default SG rules only apply to the default Security Groups, but not to the custom Security Groups?

This would be these lines in the table:

stack@devstack:~/devstack$ openstack default security group rule list --fit-width
+---------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+--------------------------------+-------------------------------+
| ID                        | IP Protocol | Ethertype | IP Range  | Port Range | Direction | Remote Security Group | Remote Address Group | Used in default Security Group | Used in custom Security Group |
+---------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+--------------------------------+-------------------------------+
| 6ace51bb-5258-45ab-9ba9-  | None        | IPv6      | ::/0      |            | ingress   | PARENT                | None                 | True                           | False                         |
| 1efbebfb086b              |             |           |           |            |           |                       |                      |                                |                               |
| 997bb0c2-652e-4d1f-b910-  | None        | IPv4      | 0.0.0.0/0 |            | ingress   | PARENT                | None                 | True                           | False                         |
| e12c89f88b44              |             |           |           |            |           |                       |                      |                                |                               |
+---------------------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+--------------------------------+-------------------------------+

The remote Security Group there is stated as "PARENT", but at least the database column says it should be and ID:

mysql> describe securitygroupdefaultrules;
+-------------------------+--------------------------+------+-----+---------+-------+
| Field                   | Type                     | Null | Key | Default | Extra |
+-------------------------+--------------------------+------+-----+---------+-------+
| id                      | varchar(36)              | NO   | PRI | NULL    |       |
| standard_attr_id        | bigint                   | NO   | UNI | NULL    |       |
| remote_group_id         | varchar(36)              | YES  |     | NULL    |       |
| remote_address_group_id | varchar(36)              | YES  |     | NULL    |       |
| direction               | enum('ingress','egress') | NO   |     | NULL    |       |
| ethertype               | varchar(40)              | YES  |     | NULL    |       |
| protocol                | varchar(40)              | YES  |     | NULL    |       |
| port_range_min          | int                      | YES  |     | NULL    |       |
| port_range_max          | int                      | YES  |     | NULL    |       |
| remote_ip_prefix        | varchar(255)             | YES  |     | NULL    |       |
| used_in_default_sg      | tinyint(1)               | NO   |     | 0       |       |
| used_in_non_default_sg  | tinyint(1)               | NO   |     | 1       |       |
+-------------------------+--------------------------+------+-----+---------+-------+

After looking through the code and trying some things I found out, that the naming "remote_group_id" is misleading for the default security group rules. If "PARENT" is stated there it automatically creates a rule for the SG that automatically puts the SGs ID as remote_group_id.

$ openstack default security group rule list
+-----------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+--------------------------------+-------------------------------+
| ID              | IP Protocol | Ethertype | IP Range  | Port Range | Direction | Remote Security Group | Remote Address Group | Used in default Security Group | Used in custom Security Group |
+-----------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+--------------------------------+-------------------------------+
| 30688079-2daa-  | None        | IPv4      | 0.0.0.0/0 |            | egress    | None                  | None                 | True                           | True                          |
| 4ad5-81e6-      |             |           |           |            |           |                       |                      |                                |                               |
| a68e725b57f4    |             |           |           |            |           |                       |                      |                                |                               |
| 47b929fd-9b39-  | None        | IPv6      | ::/0      |            | egress    | None                  | None                 | True                           | True                          |
| 4f22-abc5-      |             |           |           |            |           |                       |                      |                                |                               |
| 7d4f64d10909    |             |           |           |            |           |                       |                      |                                |                               |
| ed5cd662-add2-  | None        | IPv4      | 0.0.0.0/0 |            | ingress   | PARENT                | None                 | False                          | True                          |
| 4e42-b0a7-      |             |           |           |            |           |                       |                      |                                |                               |
| 3b585d348820    |             |           |           |            |           |                       |                      |                                |                               |
+-----------------+-------------+-----------+-----------+------------+-----------+-----------------------+----------------------+--------------------------------+-------------------------------+
stack@devstack:~/devstack$ openstack security group create new-custom-group
+-----------------+----------------------------------------------------------------------------------------------------------------------------------------+
| Field           | Value                                                                                                                                  |
+-----------------+----------------------------------------------------------------------------------------------------------------------------------------+
| created_at      | 2024-03-21T11:28:23Z                                                                                                                   |
| description     | new-custom-group                                                                                                                       |
| id              | 7dd8b73d-1448-4b2b-bb9e-c52124ee32b4                                                                                                   |
| name            | new-custom-group                                                                                                                       |
| project_id      | 15f2ab0eaa5b4372b759bde609e86224                                                                                                       |
| revision_number | 1                                                                                                                                      |
| rules           | created_at='2024-03-21T11:28:23Z', direction='egress', ethertype='IPv6', id='6f5b879d-1903-474e-a432-8d4a4e3b9a1c',                    |
|                 | standard_attr_id='160', updated_at='2024-03-21T11:28:23Z'                                                                              |
|                 | created_at='2024-03-21T11:28:23Z', direction='egress', ethertype='IPv4', id='81dc7889-a176-4f1f-861a-64463c035fea',                    |
|                 | normalized_cidr='0.0.0.0/0', remote_ip_prefix='0.0.0.0/0', standard_attr_id='159', updated_at='2024-03-21T11:28:23Z'                   |
|                 | created_at='2024-03-21T11:28:23Z', direction='ingress', ethertype='IPv4', id='e0edf57d-af2f-4c1b-9835-adb39fc2eb4a',                   |
|                 | remote_group_id='7dd8b73d-1448-4b2b-bb9e-c52124ee32b4', standard_attr_id='161', updated_at='2024-03-21T11:28:23Z'                      |
| shared          | False                                                                                                                                  |
| stateful        | True                                                                                                                                   |
| tags            | []                                                                                                                                     |
| updated_at      | 2024-03-21T11:28:23Z                                                                                                                   |
+-----------------+----------------------------------------------------------------------------------------------------------------------------------------+

That only leaves the question why in the rolled out defaults (default SG rules in an untouched new OpenStack deployment) do only apply to the default SGs but not the custom ones.

As mentioned in the SCS IAAS call I would prefer to not touch the existing default groups/rules provided by Nova/Neutron upstream. Hope that was the right place for this comment.

Looking into the topic of the ingress rules again I would like to discuss this in the IaaS meeting:

the default security group and the custom security group have the same default rules for egress traffic
the default security group has default rules that allow ingress traffic from ports with the same security group
the custom security group does not have any ingress traffic rules -> it forbids ingress at default

Additionally all security groups (default and custom) can be edited - so the default rules are just the first set of rules, which are present, when the group is created.

Questions:

Do we want to have a difference between the default rules for the default group and the custom groups?
Do we want to have the ingress rule at all, only for the default security groups or for all security groups?

A different behavior between the custom groups and the default security group may lead to confusion.

As discussed in the IaaS meeting I adjusted the Standard for the default security group rules:

They SHOULD be present as in OpenStack defaultly set. For the ingress rules I adjusted the MUST NOT to only apply to any other rules than the 2 rules allowing traffic between the default security groups.

SovereignCloudStack / standards

Add standard for default security group rules #521

Definition of Done: