tsuna-server / build-openstack-deployment

0 stars 0 forks source link

Outbound connection (floating IP) of instances don't work correctly. #23

Open TsutomuNakamura opened 1 year ago

TsutomuNakamura commented 1 year ago

If creating network ports manually (created by this Ansible), outbound connection does not work correctly. A status when it was not work correctly like below.

# openstack subnet list
+--------------------------------------+----------------+--------------------------------------+------------------+
| ID                                   | Name           | Network                              | Subnet           |
+--------------------------------------+----------------+--------------------------------------+------------------+
| 9afa430e-6f2c-49fe-87ae-01679e26b2d3 | public_subnet  | 432724c2-4d61-4e45-b6fa-9f2895c4420b | 172.31.0.0/16    |
| af85f674-1377-4f56-b077-2d9623014ad0 | private_subnet | 3ca08c7e-a0b9-4bbc-b02f-538c208c9e67 | 192.168.255.0/24 |
+--------------------------------------+----------------+--------------------------------------+------------------+

# openstack server list --long
+--------------------------------------+------------+---------+------------+-------------+-----------------------------------------+--------------+--------------------------------------+-----------+-------------------+------------------+------------+-------------+
| ID                                   | Name       | Status  | Task State | Power State | Networks                                | Image Name   | Image ID                             | Flavor    | Availability Zone | Host             | Properties | Host Status |
+--------------------------------------+------------+---------+------------+-------------+-----------------------------------------+--------------+--------------------------------------+-----------+-------------------+------------------+------------+-------------+
| 3ec56c22-f5ac-47d7-9da1-4a20f367ffca | k8s-node04 | SHUTOFF | None       | Shutdown    | private_net=172.31.1.14, 192.168.255.14 | ubuntu-22.04 | 53d68260-cf67-4fa0-a4c2-975c35608433 | m1.medium | nova              | dev-comstorage02 |            | UP          |
| a383fd26-c3b5-4d64-afec-5d8059cf105d | k8s-node03 | SHUTOFF | None       | Shutdown    | private_net=172.31.1.13, 192.168.255.13 | ubuntu-22.04 | 53d68260-cf67-4fa0-a4c2-975c35608433 | m1.medium | nova              | dev-comstorage03 |            | UP          |
| 9fc013a6-8e29-4520-a629-5097c9208b16 | k8s-node02 | SHUTOFF | None       | Shutdown    | private_net=172.31.1.12, 192.168.255.12 | ubuntu-22.04 | 53d68260-cf67-4fa0-a4c2-975c35608433 | m1.medium | nova              | dev-comstorage01 |            | UP          |
| 1d651dc4-9199-45ec-9653-ba35fc021d2b | k8s-node01 | SHUTOFF | None       | Shutdown    | private_net=172.31.1.11, 192.168.255.11 | ubuntu-22.04 | 53d68260-cf67-4fa0-a4c2-975c35608433 | m1.medium | nova              | dev-comstorage02 |            | UP          |
+--------------------------------------+------------+---------+------------+-------------+-----------------------------------------+--------------+--------------------------------------+-----------+-------------------+------------------+------------+-------------+

# openstack port list --long
+--------------------------------------+-----------------+-------------------+-------------------------------------------------------------------------------+--------+-----------------+--------------------------+------+
| ID                                   | Name            | MAC Address       | Fixed IP Addresses                                                            | Status | Security Groups | Device Owner             | Tags |
+--------------------------------------+-----------------+-------------------+-------------------------------------------------------------------------------+--------+-----------------+--------------------------+------+
| 05b5776a-4c26-4323-b395-1382f6154412 |                 | fa:16:3e:d1:f3:8c | ip_address='192.168.255.2', subnet_id='af85f674-1377-4f56-b077-2d9623014ad0'  | DOWN   | None            | network:distributed      |      |
| 097dd70f-766a-4697-a7bb-e82702e12fc1 | k8s-node04-port | fa:16:3e:a5:73:f4 | ip_address='192.168.255.14', subnet_id='af85f674-1377-4f56-b077-2d9623014ad0' | DOWN   | None            | compute:nova             |      |
| 0b5a9889-917d-444e-a0ef-388167a71eab |                 | fa:16:3e:a4:0e:8d | ip_address='172.31.230.160', subnet_id='9afa430e-6f2c-49fe-87ae-01679e26b2d3' | ACTIVE | None            | network:router_gateway   |      |
| 4dd6a22a-bab5-49d4-adce-f98d6277eb40 |                 | fa:16:3e:58:82:40 |                                                                               | DOWN   | None            | network:distributed      |      |
| 60f0c58f-6f8c-4513-b485-c6bc05ad954a | k8s-node02-port | fa:16:3e:e9:e7:fe | ip_address='192.168.255.12', subnet_id='af85f674-1377-4f56-b077-2d9623014ad0' | DOWN   | None            | compute:nova             |      |
| 6dcc4ffe-74f2-48ad-ac24-ceea15d7689d | k8s-node03-port | fa:16:3e:e4:35:5f | ip_address='192.168.255.13', subnet_id='af85f674-1377-4f56-b077-2d9623014ad0' | DOWN   | None            | compute:nova             |      |
| 71d9ca6e-449a-447b-8477-e07c50ca3158 |                 | fa:16:3e:df:08:ec | ip_address='172.31.1.11', subnet_id='9afa430e-6f2c-49fe-87ae-01679e26b2d3'    | N/A    | None            | network:floatingip       |      |
| 7cfaf9fb-c271-44dd-8363-e407d7c51ded |                 | fa:16:3e:c7:32:b1 | ip_address='192.168.255.1', subnet_id='af85f674-1377-4f56-b077-2d9623014ad0'  | ACTIVE | None            | network:router_interface |      |
| 95b29a78-0433-4573-bc69-3e5444979de4 | k8s-node01-port | fa:16:3e:4e:b3:59 | ip_address='192.168.255.11', subnet_id='af85f674-1377-4f56-b077-2d9623014ad0' | DOWN   | None            | compute:nova             |      |
| 9a5fe4fd-0d89-4253-a626-bfdaa33b45f2 |                 | fa:16:3e:91:e1:03 | ip_address='172.31.1.14', subnet_id='9afa430e-6f2c-49fe-87ae-01679e26b2d3'    | N/A    | None            | network:floatingip       |      |
| 9d1008cf-76f3-4aae-9fd0-cf430871211e |                 | fa:16:3e:42:b8:30 | ip_address='172.31.1.13', subnet_id='9afa430e-6f2c-49fe-87ae-01679e26b2d3'    | N/A    | None            | network:floatingip       |      |
| a51e1bb5-012f-4b0e-9bd2-ec39dd846e7f |                 | fa:16:3e:c1:43:79 | ip_address='172.31.1.12', subnet_id='9afa430e-6f2c-49fe-87ae-01679e26b2d3'    | N/A    | None            | network:floatingip       |      |
+--------------------------------------+-----------------+-------------------+-------------------------------------------------------------------------------+--------+-----------------+--------------------------+------+

I want to see a difference of outputs in a correct situation and wrong situation.

Obtained additional info.

# openstack port show 4b65208f-b174-4156-9ec4-6097510ee817
+-------------------------+-------------------------------------------------------------------------------+
| Field                   | Value                                                                         |
+-------------------------+-------------------------------------------------------------------------------+
| admin_state_up          | UP                                                                            |
| allowed_address_pairs   |                                                                               |
| binding_host_id         | dev-comstorage02                                                              |
| binding_profile         |                                                                               |
| binding_vif_details     | bound_drivers.0='ovn', connectivity='l2', port_filter='True'                  |
| binding_vif_type        | ovs                                                                           |
| binding_vnic_type       | normal                                                                        |
| created_at              | 2023-10-07T03:21:33Z                                                          |
| data_plane_status       | None                                                                          |
| description             |                                                                               |
| device_id               | fecc24f1-f943-41ae-967c-804bcc9c4c90                                          |
| device_owner            | compute:nova                                                                  |
| device_profile          | None                                                                          |
| dns_assignment          | None                                                                          |
| dns_domain              | None                                                                          |
| dns_name                | None                                                                          |
| extra_dhcp_opts         |                                                                               |
| fixed_ips               | ip_address='192.168.255.11', subnet_id='ac2cc78b-10b8-4651-a8fc-1ad5ac93e4a8' |
| id                      | 4b65208f-b174-4156-9ec4-6097510ee817                                          |
| ip_allocation           | None                                                                          |
| mac_address             | fa:16:3e:f3:8d:17                                                             |
| name                    | k8s-node01-port                                                               |
| network_id              | 75a9c3b9-141b-435c-a58f-9234400db237                                          |
| numa_affinity_policy    | None                                                                          |
| port_security_enabled   | True                                                                          |
| project_id              | fe5e2f00965845daa6e3bbc6714dd791                                              |
| propagate_uplink_status | None                                                                          |
| qos_network_policy_id   | None                                                                          |
| qos_policy_id           | None                                                                          |
| resource_request        | None                                                                          |
| revision_number         | 4                                                                             |
| security_group_ids      | 0a46a731-2620-4ddc-b394-8b9c65685ff1                                          |
| status                  | ACTIVE                                                                        |
| tags                    |                                                                               |
| trunk_details           | None                                                                          |
| updated_at              | 2023-10-07T03:22:02Z                                                          |
+-------------------------+-------------------------------------------------------------------------------+
# openstack subnet show ac2cc78b-10b8-4651-a8fc-1ad5ac93e4a8
+----------------------+--------------------------------------+
| Field                | Value                                |
+----------------------+--------------------------------------+
| allocation_pools     | 192.168.255.2-192.168.255.254        |
| cidr                 | 192.168.255.0/24                     |
| created_at           | 2023-10-07T03:20:56Z                 |
| description          |                                      |
| dns_nameservers      | 172.31.0.1, 8.8.4.4, 8.8.8.8         |
| dns_publish_fixed_ip | None                                 |
| enable_dhcp          | True                                 |
| gateway_ip           | 192.168.255.1                        |
| host_routes          |                                      |
| id                   | ac2cc78b-10b8-4651-a8fc-1ad5ac93e4a8 |
| ip_version           | 4                                    |
| ipv6_address_mode    | None                                 |
| ipv6_ra_mode         | None                                 |
| name                 | private_subnet                       |
| network_id           | 75a9c3b9-141b-435c-a58f-9234400db237 |
| project_id           | fe5e2f00965845daa6e3bbc6714dd791     |
| revision_number      | 0                                    |
| segment_id           | None                                 |
| service_types        |                                      |
| subnetpool_id        | None                                 |
| tags                 |                                      |
| updated_at           | 2023-10-07T03:20:56Z                 |
+----------------------+--------------------------------------+
TsutomuNakamura commented 1 year ago

When the instructions creating ports was removed, the connections will recovered.

# openstack subnet list
+--------------------------------------+----------------+--------------------------------------+------------------+
| ID                                   | Name           | Network                              | Subnet           |
+--------------------------------------+----------------+--------------------------------------+------------------+
| 0fdef627-4e93-40c7-8a7e-5f6bc7cf90c5 | public_subnet  | 9f6ea511-94ae-4fd9-95cc-2f025823cba9 | 172.31.0.0/16    |
| 607fa537-446f-47b5-9b06-5f9a28171bae | private_subnet | ad771c4e-dd09-4bf4-9624-5633f5f46127 | 192.168.255.0/24 |
+--------------------------------------+----------------+--------------------------------------+------------------+

# openstack server list --long
+--------------------------------------+------------+--------+------------+-------------+------------------------------------------+--------------+--------------------------------------+-----------+-------------------+------------------+------------+-------------+
| ID                                   | Name       | Status | Task State | Power State | Networks                                 | Image Name   | Image ID                             | Flavor    | Availability Zone | Host             | Properties | Host Status |
+--------------------------------------+------------+--------+------------+-------------+------------------------------------------+--------------+--------------------------------------+-----------+-------------------+------------------+------------+-------------+
| b1c554e4-fcae-4889-a4db-ff4b75aad40d | k8s-node04 | ACTIVE | None       | Running     | private_net=172.31.1.14, 192.168.255.152 | ubuntu-22.04 | e2797386-91cd-46e2-8ded-b27c8a83761c | m1.medium | nova              | dev-comstorage02 |            | UP          |
| 60007b5c-b4a6-4e4f-ac43-7af6f9de9b67 | k8s-node03 | ACTIVE | None       | Running     | private_net=172.31.1.13, 192.168.255.104 | ubuntu-22.04 | e2797386-91cd-46e2-8ded-b27c8a83761c | m1.medium | nova              | dev-comstorage01 |            | UP          |
| 7c942594-f351-44b9-929c-ff9af5e1ea78 | k8s-node02 | ACTIVE | None       | Running     | private_net=172.31.1.12, 192.168.255.79  | ubuntu-22.04 | e2797386-91cd-46e2-8ded-b27c8a83761c | m1.medium | nova              | dev-comstorage03 |            | UP          |
| 18c3fee9-d9f7-4146-83a6-839348cff357 | k8s-node01 | ACTIVE | None       | Running     | private_net=172.31.1.11, 192.168.255.239 | ubuntu-22.04 | e2797386-91cd-46e2-8ded-b27c8a83761c | m1.medium | nova              | dev-comstorage02 |            | UP          |
+--------------------------------------+------------+--------+------------+-------------+------------------------------------------+--------------+--------------------------------------+-----------+-------------------+------------------+------------+-------------+

# openstack port list --long
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------+--------+-----------------+--------------------------+------+
| ID                                   | Name | MAC Address       | Fixed IP Addresses                                                             | Status | Security Groups | Device Owner             | Tags |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------+--------+-----------------+--------------------------+------+
| 08e5b25d-d185-4fa1-99bc-a7605d0c9b44 |      | fa:16:3e:26:c9:58 | ip_address='192.168.255.1', subnet_id='607fa537-446f-47b5-9b06-5f9a28171bae'   | ACTIVE | None            | network:router_interface |      |
| 0f42962c-2a38-4b7f-b520-31a5839c2529 |      | fa:16:3e:32:8a:a7 | ip_address='192.168.255.104', subnet_id='607fa537-446f-47b5-9b06-5f9a28171bae' | ACTIVE | None            | compute:nova             |      |
| 361b7707-f7e9-4804-aeac-86cbbc35e054 |      | fa:16:3e:3a:15:1a | ip_address='172.31.1.12', subnet_id='0fdef627-4e93-40c7-8a7e-5f6bc7cf90c5'     | N/A    | None            | network:floatingip       |      |
| 4085912e-a62c-4229-aa09-667c2a7ce56f |      | fa:16:3e:3c:1c:19 | ip_address='172.31.1.14', subnet_id='0fdef627-4e93-40c7-8a7e-5f6bc7cf90c5'     | N/A    | None            | network:floatingip       |      |
| 7b4fc9e4-c401-4489-8661-82d13b164cb8 |      | fa:16:3e:6a:eb:aa |                                                                                | DOWN   | None            | network:distributed      |      |
| 928f4de3-7951-462b-b9d7-48f7b320eeba |      | fa:16:3e:c8:3d:a9 | ip_address='172.31.230.170', subnet_id='0fdef627-4e93-40c7-8a7e-5f6bc7cf90c5'  | ACTIVE | None            | network:router_gateway   |      |
| 94b6de11-e516-4129-965a-8e706a6ccca4 |      | fa:16:3e:e6:32:a7 | ip_address='172.31.1.13', subnet_id='0fdef627-4e93-40c7-8a7e-5f6bc7cf90c5'     | N/A    | None            | network:floatingip       |      |
| a7116314-9a34-494c-919a-699533802c0f |      | fa:16:3e:9c:77:2a | ip_address='192.168.255.2', subnet_id='607fa537-446f-47b5-9b06-5f9a28171bae'   | DOWN   | None            | network:distributed      |      |
| d0d3258f-871b-4d0c-b95e-bc6222ae4833 |      | fa:16:3e:f4:bc:7f | ip_address='192.168.255.79', subnet_id='607fa537-446f-47b5-9b06-5f9a28171bae'  | ACTIVE | None            | compute:nova             |      |
| db80a06e-ff77-4152-8061-cbb8aa7ae068 |      | fa:16:3e:26:ca:3f | ip_address='172.31.1.11', subnet_id='0fdef627-4e93-40c7-8a7e-5f6bc7cf90c5'     | N/A    | None            | network:floatingip       |      |
| ea433542-1da1-4b20-8c9a-b51c905a56e0 |      | fa:16:3e:a9:88:64 | ip_address='192.168.255.152', subnet_id='607fa537-446f-47b5-9b06-5f9a28171bae' | ACTIVE | None            | compute:nova             |      |
| f34168ae-b7eb-4c8b-864f-b064fe5e309d |      | fa:16:3e:d9:92:85 | ip_address='192.168.255.239', subnet_id='607fa537-446f-47b5-9b06-5f9a28171bae' | ACTIVE | None            | compute:nova             |      |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------+--------+-----------------+--------------------------+------+
root@dev-controller01:~# openstack port show 18ccd998-5ffc-4414-8905-61adb10997c3
+-------------------------+-------------------------------------------------------------------------------+
| Field                   | Value                                                                         |
+-------------------------+-------------------------------------------------------------------------------+
| admin_state_up          | UP                                                                            |
| allowed_address_pairs   |                                                                               |
| binding_host_id         | dev-comstorage01                                                              |
| binding_profile         |                                                                               |
| binding_vif_details     | bound_drivers.0='ovn', connectivity='l2', port_filter='True'                  |
| binding_vif_type        | ovs                                                                           |
| binding_vnic_type       | normal                                                                        |
| created_at              | 2023-10-07T06:23:25Z                                                          |
| data_plane_status       | None                                                                          |
| description             |                                                                               |
| device_id               | ffa706e8-1776-4b59-b1d6-2602a2e77115                                          |
| device_owner            | compute:nova                                                                  |
| device_profile          | None                                                                          |
| dns_assignment          | None                                                                          |
| dns_domain              | None                                                                          |
| dns_name                | None                                                                          |
| extra_dhcp_opts         |                                                                               |
| fixed_ips               | ip_address='192.168.255.77', subnet_id='12718a02-9cc2-4675-b514-65a276b5dc9a' |
| id                      | 18ccd998-5ffc-4414-8905-61adb10997c3                                          |
| ip_allocation           | None                                                                          |
| mac_address             | fa:16:3e:e6:6a:9c                                                             |
| name                    |                                                                               |
| network_id              | dcbabda1-c331-4d88-accf-ee71eca5cfdc                                          |
| numa_affinity_policy    | None                                                                          |
| port_security_enabled   | True                                                                          |
| project_id              | fe5e2f00965845daa6e3bbc6714dd791                                              |
| propagate_uplink_status | None                                                                          |
| qos_network_policy_id   | None                                                                          |
| qos_policy_id           | None                                                                          |
| resource_request        | None                                                                          |
| revision_number         | 4                                                                             |
| security_group_ids      | b644f1d8-dce3-46fd-81db-0414f9d03b47                                          |
| status                  | ACTIVE                                                                        |
| tags                    |                                                                               |
| trunk_details           | None                                                                          |
| updated_at              | 2023-10-07T06:23:37Z                                                          |
+-------------------------+-------------------------------------------------------------------------------+

root@dev-controller01:~# openstack subnet show 12718a02-9cc2-4675-b514-65a276b5dc9a
+----------------------+--------------------------------------+
| Field                | Value                                |
+----------------------+--------------------------------------+
| allocation_pools     | 192.168.255.2-192.168.255.254        |
| cidr                 | 192.168.255.0/24                     |
| created_at           | 2023-10-07T06:22:22Z                 |
| description          |                                      |
| dns_nameservers      | 172.31.0.1, 8.8.4.4, 8.8.8.8         |
| dns_publish_fixed_ip | None                                 |
| enable_dhcp          | True                                 |
| gateway_ip           | 192.168.255.1                        |
| host_routes          |                                      |
| id                   | 12718a02-9cc2-4675-b514-65a276b5dc9a |
| ip_version           | 4                                    |
| ipv6_address_mode    | None                                 |
| ipv6_ra_mode         | None                                 |
| name                 | private_subnet                       |
| network_id           | dcbabda1-c331-4d88-accf-ee71eca5cfdc |
| project_id           | fe5e2f00965845daa6e3bbc6714dd791     |
| revision_number      | 0                                    |
| segment_id           | None                                 |
| service_types        |                                      |
| subnetpool_id        | None                                 |
| tags                 |                                      |
| updated_at           | 2023-10-07T06:22:22Z                 |
+----------------------+--------------------------------------+

# openstack security group list
+--------------------------------------+------------+------------------------+----------------------------------+------+
| ID                                   | Name       | Description            | Project                          | Tags |
+--------------------------------------+------------+------------------------+----------------------------------+------+
| 774020fa-0564-456d-92ab-428ec74f79bf | default    | Default security group | fe5e2f00965845daa6e3bbc6714dd791 | []   |
| b644f1d8-dce3-46fd-81db-0414f9d03b47 | permit_all | Permit all packets     | fe5e2f00965845daa6e3bbc6714dd791 | []   |
| e30e1c35-d6f1-47c7-9293-2e977ead73d9 | default    | Default security group | 3793062575c4427faf1d0f8e74efc98d | []   |
+--------------------------------------+------------+------------------------+----------------------------------+------+
TsutomuNakamura commented 1 year ago

Should ports are activated when they were created by manually?