search

futurewei-cloud / alcor

Alcor: Cloud native SDN platform powered by Kubernetes and Istio
MIT License
32 stars 33 forks source link

Port Manager (port 9006) creates empty neighborTable when multiple subnets used #589

Open kiran1048 opened 3 years ago

kiran1048 commented 3 years ago

Reproduce:

  1. Create 2 subnets in the alcor services before creating goal states
  2. Create a router interface to connect the subnets
  3. Create 2 goal states with, one with each subnet.

Expected: The two goal states should have neighborInfo and neighborTable with information about other goal state.

Observed: The neighborTable in the port manager container logs show up empty. The logs pasted below: Network configuration: Netwo rkConfiguration{rsType=PORT, opType=null, portEntities=[InternalPortEntity(routes=[], bindingHostIP=10.213.43.163, isZetaGate wayPort=false, internalNeighborInfo1=null, subnetEntities=null, vpcEntities=null)], vpcs=[VpcEntity(cidr=10.0.0.0/16, routeEn tities=[RouteEntity(destination=10.0.0.0/16, target=Local, priority=0, associatedType=VPC, associatedTableId=f579396a-bebf-4e 6c-b30f-ee17e8544e05)], router=Router(neutronRouteTable=null, neutronSubnetRouteTables=null, vpcRouteTables=null, vpcDefaultR outeTableId=null, owner=null, routerExtraAttributeId=null, gatewayPorts=null, tenantId=null, adminStateUp=false, status=null, created_at=null, updated_at=null), adminStateUp=true, dnsDomain=domain, mtu=1400, portSecurityEnabled=true, networkType=vxla n, physicalNetwork=, segmentationId=3973120, routerExternal=false, segments=[], shared=false, vlanTransparent=false, isDefaul t=true, availabilityZoneHints=[], availabilityZones=[Nova], qosPolicyId=, revisionNumber=1, status=ACTIVE, tags=[], created_a t=2021-04-08 22:40:08, updated_at=2021-04-08 22:40:08, ipv4AddressScope=, ipv6AddressScope=, l2Adjacency=, subnets=[8182a4d4- ffff-4ece-b3f0-8d36e3d88001, 8182a4d4-ffff-4ece-b3f0-8d36e3d88002])], subnets=[InternalSubnetEntity(tunnelId=3973120)], secur ityGroups=[CustomerResource{projectId='3dda2801-d675-4688-a63f-dcda8d327f50', id='3dda2801-d675-4688-a63f-dcda8d111111', name ='sg1', description='string'}], neighborInfos={10.0.1.101=NeighborInfo(hostIp=10.213.43.161, hostId=node1, portId=7172a4d4-ff ff-4ede-b3ml0-8d36e3d00101, portMac=aa:bb:cc:2a:2a:da, portIp=10.0.1.101, vpcId=9192a4d4-ffff-4ece-b3f0-8d36e3d88001, subnetId=8182a4d4-ffff-4ece-b3f0-8d36e3d88001)}, neighborTable={10.0.2.101=[]}, routerInfo=null}

kiran1048 commented 3 years ago

During the test, the two compute nodes were successfully assigned the following goal states: Container1: ipaddress=10.0.1.101/24 macaddress=aa:bb:cc:4e:d6:5a Container2: ipaddress=10.0.2.101/24 macaddress=aa:bb:cc:3d:14:21

The two subnets created are: subnet1: 8182a4d4-ffff-4ece-b3f0-8d36e3d88001 subnet2: 8182a4d4-ffff-4ece-b3f0-8d36e3d88002

Through an alcor API call we create an interface between the above two subnets. The output of the API call is:

{"id":"11112801-d675-4688-a63f-dcda8d327f50","network_id":"9192a4d4-ffff-4ece-b3f0-8d36e3d88001","port_id":"12149c33-1f5f-41c8-8108-c2b06c42d931","subnet_id":"8182 a4d4-ffff-4ece-b3f0-8d36e3d88001","subnet_ids":["8182a4d4-ffff-4ece-b3f0-8d36e3d88001"],"project_id":"3dda2801-d675-4688-a63f-dcda8d327f50","tenant_id":"3dda2801-d 675-4688-a63f-dcda8d327f50","tags":[]}{"id":"11112801-d675-4688-a63f-dcda8d327f50","network_id":"9192a4d4-ffff-4ece-b3f0-8d36e3d88001","port_id":"629b5f28-4a07-459 9-b1a2-fa9624472f11","subnet_id":"8182a4d4-ffff-4ece-b3f0-8d36e3d88002","subnet_ids":["8182a4d4-ffff-4ece-b3f0-8d36e3d88002"],"project_id":"3dda2801-d675-4688-a63f -dcda8d327f50","tenant_id":"3dda2801-d675-4688-a63f-dcda8d327f50","tags":[]}

After successful creation of goal states, we have following logs from PM. We can notice that neighbor table is empty:

2021-04-09 22:21:08.598 INFO 1 --- [io-9006-exec-10] c.f.a.p.processor.DataPlaneProcessor : Network configuration: NetworkConfiguration{rsType=PORT, opType=null, portEntities=[InternalPortEntity(routes=[], bindingHostIP=10.213.43.163, isZetaGatewayPort=false, internalNeighborInfo1=null, subnetEntities=null, vpcEntities=null)], vpcs=[VpcEntity(cidr=10.0.0.0/16, routeEntities=[RouteEntity(destination=10.0.0.0/16, target=Local, priority=0, associatedType=VPC, associatedTableId=1984ee4e-f519-4b0d-814f-16c01c3e29d7)], router=Router(neutronRouteTable=null, neutronSubnetRouteTables=null, vpcRouteTables=null, vpcDefaultRouteTableId=null, owner=null, routerExtraAttributeId=null, gatewayPorts=null, tenantId=null, adminStateUp=false, status=null, created_at=null, updated_at=null), adminStateUp=true, dnsDomain=domain, mtu=1400, portSecurityEnabled=true, networkType=vxlan, physicalNetwork=, segmentationId=8716288, routerExternal=false, segments=[], shared=false, vlanTransparent=false, isDefault=true, availabilityZoneHints=[], availabilityZones=[Nova], qosPolicyId=, revisionNumber=1, status=ACTIVE, tags=[], created_at=2021-04-09 22:12:45, updated_at=2021-04-09 22:12:45, ipv4AddressScope=, ipv6AddressScope=, l2Adjacency=, subnets=[8182a4d4-ffff-4ece-b3f0-8d36e3d88001, 8182a4d4-ffff-4ece-b3f0-8d36e3d88002])], subnets=[InternalSubnetEntity(tunnelId=8716288)], securityGroups=[CustomerResource{projectId='3dda2801-d675-4688-a63f-dcda8d327f50', id='3dda2801-d675-4688-a63f-dcda8d111111', name='sg1', description='string'}], neighborInfos={10.0.1.101=NeighborInfo(hostIp=10.213.43.161, hostId=node1, portId=7172a4d4-ffff-4ede-b3ml0-8d36e3d00101, portMac=aa:bb:cc:4e:d6:5a, portIp=10.0.1.101, vpcId=9192a4d4-ffff-4ece-b3f0-8d36e3d88001, subnetId=8182a4d4-ffff-4ece-b3f0-8d36e3d88001)}, neighborTable={10.0.2.101=[]}, routerInfo=null}

The corresponding docker logs from DPM are:

{"rsType":"PORT","opType":"CREATE","ports_internal":[{"bindingHostId":"node2","id":"7172a4d4-ffff-4ede-b3ml0-8d36e3d00102","project_id":"3dda2801-d675-4688-a63f-dc da8d327f50","tenant_id":"3dda2801-d675-4688-a63f-dcda8d327f50","name":"port201","description":"string","network_id":"9192a4d4-ffff-4ece-b3f0-8d36e3d88001","admin_s tate_up":true,"mac_address":"aa:bb:cc:3d:14:21","veth_name":"string","fast_path":true,"device_id":"8182a4d4-ffff-4ece-b3f0-8d36e3d00002","device_owner":"compute:no va","fixed_ips":[{"subnet_id":"8182a4d4-ffff-4ece-b3f0-8d36e3d88002","ip_address":"10.0.2.101"}],"allowed_address_pairs":[{"ip_address":"11.11.11.1","mac_address": "00-AA-BB-15-EB-3F"}],"extra_dhcp_opts":[{"ip_version":"string","opt_name":"string","opt_value":"string"}],"security_groups":["3dda2801-d675-4688-a63f-dcda8d111111 "],"binding:vif_details":{"ovs_hybrid_plug":false,"port_filter":false},"binding:vif_type":"ovs","network_ns":"string","dns_name":"string","dns_domain":"string","dn s_assignment":{"host_name":null,"ip_address":null,"fqdn":null},"create_at":"string","update_at":"string","port_security_enabled":true,"qos_network_policy_id":"stri ng","qos_policy_id":"string","revision_number":0,"tags":["string"],"uplink_status_propagation":true,"mac_learning_enabled":true,"routes":[],"binding_host_ip":"10.2 13.43.163","is_zeta_gateway_port":false}],"vpcs_internal":[{"default":true,"id":"9192a4d4-ffff-4ece-b3f0-8d36e3d88001","project_id":"3dda2801-d675-4688-a63f-dcda8d 327f50","tenant_id":"3dda2801-d675-4688-a63f-dcda8d327f50","name":"sample_vpc","description":"vpc","cidr":"10.0.0.0/16","routes":[{"destination":"10.0.0.0/16","tar get":"Local","priority":0,"associatedType":"VPC","associatedTableId":"1984ee4e-f519-4b0d-814f-16c01c3e29d7","id":"1a21e380-103a-48e4-bcaa-e5ef4b17417a","project_id ":"3dda2801-d675-4688-a63f-dcda8d327f50","tenant_id":"3dda2801-d675-4688-a63f-dcda8d327f50","name":"default_route_rule","description":""}],"router":{"id":null,"pro ject_id":null,"tenant_id":null,"name":null,"description":null,"neutron_router_routetable":null,"neutron_subnet_routetables":null,"routetables":null,"vpc_default_ro ute_table_id":null,"owner":null,"router_extra_attribute_id":null,"gateway_ports":null,"admin_state_up":false,"status":null,"created_at":null,"updated_at":null},"ad min_state_up":true,"dns_domain":"domain","mtu":1400,"port_security_enabled":true,"provider:network_type":"vxlan","provider:physical_network":"","provider:segmentat ion_id":8716288,"router:external":false,"segments":[],"shared":false,"vlan_transparent":false,"is_default":true,"availability_zone_hints":[],"availability_zones":[ "Nova"],"qos_policy_id":"","revision_number":1,"status":"ACTIVE","tags":[],"created_at":"2021-04-09 22:12:45","updated_at":"2021-04-09 22:12:45","ipv4_address_scop e":"","ipv6_address_scope":"","l2_adjacency":"","subnets":["8182a4d4-ffff-4ece-b3f0-8d36e3d88001","8182a4d4-ffff-4ece-b3f0-8d36e3d88002"]}],"subnets_internal":[{"i d":"8182a4d4-ffff-4ece-b3f0-8d36e3d88002","project_id":"3dda2801-d675-4688-a63f-dcda8d327f50","tenant_id":"3dda2801-d675-4688-a63f-dcda8d327f50","name":"subnet2"," description":null,"network_id":"9192a4d4-ffff-4ece-b3f0-8d36e3d88001","cidr":"10.0.2.0/24","availability_zone":null,"gateway_ip":"10.0.2.1","gatewayPortId":null,"g ateway_port_detail":{"gateway_macAddress":"aa:bb:cc:fa:45:3e","gateway_port_id":"629b5f28-4a07-4599-b1a2-fa9624472f11"},"attached_router_id":null,"port_detail":nul l,"enable_dhcp":true,"primary_dns":null,"secondary_dns":null,"dns_list":null,"ip_version":4,"ipV4_rangeId":"920b5b50-f6b5-491e-8d09-d5513673a5d9","ipV6_rangeId":nu ll,"ipv6_address_mode":null,"ipv6_ra_mode":null,"revision_number":2,"segment_id":null,"shared":null,"sort_dir":null,"sort_key":null,"subnetpool_id":null,"dns_publi sh_fixed_ip":false,"tags":[],"tags-any":null,"not-tags":null,"not-tags-any":null,"fields":null,"dns_nameservers":[],"allocation_pools":[{"start":"10.0.2.1","end":" 10.0.2.254"}],"host_routes":[{"destination":"10.0.1.0/24","nexthop":"10.0.2.1"}],"prefixlen":null,"use_default_subnet_pool":false,"service_types":[],"created_at":" 2021-04-09 22:16:48","updated_at":"2021-04-09 22:16:48","tunnel_id":8716288}],"security_groups_internal":[{"id":"3dda2801-d675-4688-a63f-dcda8d111111","project_id" :"3dda2801-d675-4688-a63f-dcda8d327f50","tenant_id":"3dda2801-d675-4688-a63f-dcda8d327f50","name":"sg1","description":"string","security_group_rules":[{"id":"b9e4a cc3-8103-4923-9ad6-9401960b210b","project_id":null,"tenant_id":"3dda2801-d675-4688-a63f-dcda8d327f50","name":null,"description":null,"security_group_id":"3dda2801- d675-4688-a63f-dcda8d111111","remote_group_id":null,"direction":"egress","remote_ip_prefix":null,"protocol":null,"port_range_max":null,"port_range_min":null,"ether type":"IPv4"},{"id":"c06f70a3-0d62-4d1f-8b59-36563550b864","project_id":null,"tenant_id":"3dda2801-d675-4688-a63f-dcda8d327f50","name":null,"description":null,"sec urity_group_id":"3dda2801-d675-4688-a63f-dcda8d111111","remote_group_id":null,"direction":"egress","remote_ip_prefix":null,"protocol":null,"port_range_max":null,"p ort_range_min":null,"ethertype":"IPv6"}],"create_at":"2021-04-09 22:18:32","update_at":"2021-04-09 22:18:32"}],"neighbor_info":{"10.0.1.101":{"host_ip":"10.213.43. 161","host_id":"node1","port_id":"7172a4d4-ffff-4ede-b3ml0-8d36e3d00101","port_mac":"aa:bb:cc:4e:d6:5a","port_ip":"10.0.1.101","vpc_id":"9192a4d4-ffff-4ece-b3f0-8d 36e3d88001","subnet_id":"8182a4d4-ffff-4ece-b3f0-8d36e3d88001"}},"neighbor_table":{"10.0.2.101":[]},"routers_internal":null}]

vpc_states { configuration { revision_number: 1 id: "9192a4d4-ffff-4ece-b3f0-8d36e3d88001" project_id: "3dda2801-d675-4688-a63f-dcda8d327f50" name: "sample_vpc" cidr: "10.0.0.0/16" } } subnet_states { configuration { revision_number: 1 id: "8182a4d4-ffff-4ece-b3f0-8d36e3d88002" vpc_id: "9192a4d4-ffff-4ece-b3f0-8d36e3d88001" name: "subnet2" cidr: "10.0.2.0/24" tunnel_id: 8716288 gateway { ip_address: "10.0.2.1" mac_address: "aa:bb:cc:fa:45:3e" } dhcp_enable: true } } port_states { operation_type: CREATE configuration { revision_number: 1 id: "7172a4d4-ffff-4ede-b3ml0-8d36e3d00102" update_type: FULL vpc_id: "9192a4d4-ffff-4ece-b3f0-8d36e3d88001" name: "port201" device_id: "8182a4d4-ffff-4ece-b3f0-8d36e3d00002" device_owner: "compute:nova" mac_address: "aa:bb:cc:3d:14:21" admin_state_up: true host_info { ip_address: "10.213.43.163" } fixed_ips { subnet_id: "8182a4d4-ffff-4ece-b3f0-8d36e3d88002" ip_address: "10.0.2.101" } allow_address_pairs { ip_address: "11.11.11.1" mac_address: "00-AA-BB-15-EB-3F" } security_group_ids { id: "3dda2801-d675-4688-a63f-dcda8d111111" } } } security_group_states { operation_type: CREATE configuration { revision_number: 1 id: "3dda2801-d675-4688-a63f-dcda8d111111" name: "sg1" security_group_rules { security_group_id: "3dda2801-d675-4688-a63f-dcda8d111111" id: "b9e4acc3-8103-4923-9ad6-9401960b210b" } security_group_rules { security_group_id: "3dda2801-d675-4688-a63f-dcda8d111111" id: "c06f70a3-0d62-4d1f-8b59-36563550b864" ethertype: IPV6 } } } dhcp_states { operation_type: CREATE configuration { revision_number: 1 mac_address: "aa:bb:cc:3d:14:21" ipv4_address: "10.0.2.101" } }