sonic-net / DASH

Disaggregated APIs for SONiC Hosts
Apache License 2.0
77 stars 88 forks source link

why is overlay dest mac 0000000000 valid for outbound traffic for BMv2 #583

Open mgheorghe opened 1 month ago

mgheorghe commented 1 month ago

why dest mac can be 00000000 ?

i looked at https://github.com/sonic-net/DASH/blob/main/documentation/general/dash-sai-pipeline-packet-flow.md#59-matching-stages-and-metadata-publishing and i'm not clear if dst mac needs to match the "outbound_ca_topa#eni1nsg0acl0i0A"

overlay_eth1.src.value = eni_mac
overlay_eth1.dst.value = '00:00:00:00:00:00'

overlay_ip1.src.value = eni_ip
overlay_ip1.dst.value = remote_ip_a_eni
[
    {"name": "vip_#1", "op": "create", "type": "SAI_OBJECT_TYPE_VIP_ENTRY", "key": {"switch_id": "$SWITCH_ID", "vip": "221.0.0.1"}, "attributes": ["SAI_VIP_ENTRY_ATTR_ACTION", "SAI_VIP_ENTRY_ACTION_ACCEPT"]}, 
    {"name": "direction_lookup_entry_#eni1", "op": "create", "type": "SAI_OBJECT_TYPE_DIRECTION_LOOKUP_ENTRY", "key": {"switch_id": "$SWITCH_ID", "vni": "1"}, "attributes": ["SAI_DIRECTION_LOOKUP_ENTRY_ATTR_ACTION", "SAI_DIRECTION_LOOKUP_ENTRY_ACTION_SET_OUTBOUND_DIRECTION"]}, 
   {"name": "vnet_#eni1", "op": "create", "type": "SAI_OBJECT_TYPE_VNET", "attributes": ["SAI_VNET_ATTR_VNI", "1001"]}, 

    {
        "name": "eni_#1", 
        "op": "create", 
        "type": "SAI_OBJECT_TYPE_ENI", 
        "attributes": [
            "SAI_ENI_ATTR_CPS", "0", "SAI_ENI_ATTR_PPS", "0", "SAI_ENI_ATTR_FLOWS", "0", "SAI_ENI_ATTR_ADMIN_STATE", "True", "SAI_ENI_ATTR_HA_SCOPE_ID", "0", "SAI_ENI_ATTR_VM_UNDERLAY_DIP", "221.1.0.0", "SAI_ENI_ATTR_VM_VNI", "1", "SAI_ENI_ATTR_VNET_ID", "$vnet_#eni1", "SAI_ENI_ATTR_INBOUND_V4_STAGE1_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_INBOUND_V4_STAGE2_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_INBOUND_V4_STAGE3_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_INBOUND_V4_STAGE4_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_INBOUND_V4_STAGE5_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_INBOUND_V6_STAGE1_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_INBOUND_V6_STAGE2_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_INBOUND_V6_STAGE3_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_INBOUND_V6_STAGE4_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_INBOUND_V6_STAGE5_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_OUTBOUND_V4_STAGE1_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_OUTBOUND_V4_STAGE2_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_OUTBOUND_V4_STAGE3_DASH_ACL_GROUP_ID", "0", 
            "SAI_ENI_ATTR_OUTBOUND_V4_STAGE4_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_OUTBOUND_V4_STAGE5_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_OUTBOUND_V6_STAGE1_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_OUTBOUND_V6_STAGE2_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_OUTBOUND_V6_STAGE3_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_OUTBOUND_V6_STAGE4_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_OUTBOUND_V6_STAGE5_DASH_ACL_GROUP_ID", "0", "SAI_ENI_ATTR_V4_METER_POLICY_ID", "0", "SAI_ENI_ATTR_V6_METER_POLICY_ID", "0"
        ]
    }, 

    {"name": "eni_ether_address_map_#eni1", "op": "create", "type": "SAI_OBJECT_TYPE_ENI_ETHER_ADDRESS_MAP_ENTRY", "key": {"switch_id": "$SWITCH_ID", "address": "00:1A:C5:00:00:01"}, "attributes": ["SAI_ENI_ETHER_ADDRESS_MAP_ENTRY_ATTR_ENI_ID", "$eni_#1"]}, 
   {"name": "outbound_ca_to_pa_#eni1nsg0acl0i0A", "op": "create", "type": "SAI_OBJECT_TYPE_OUTBOUND_CA_TO_PA_ENTRY", "key": {"switch_id": "$SWITCH_ID", "dst_vnet_id": "$vnet_#eni1", "dip": "1.4.0.1"}, "attributes": ["SAI_OUTBOUND_CA_TO_PA_ENTRY_ATTR_UNDERLAY_DIP", "221.2.0.0", "SAI_OUTBOUND_CA_TO_PA_ENTRY_ATTR_OVERLAY_DMAC", "00:1B:6E:00:00:01", "SAI_OUTBOUND_CA_TO_PA_ENTRY_ATTR_USE_DST_VNET_VNI", "True"]}, 
    {"name": "outbound_routing_#eni1nsg0acl0ri0", "op": "create", "type": "SAI_OBJECT_TYPE_OUTBOUND_ROUTING_ENTRY", "key": {"switch_id": "$SWITCH_ID", "eni_id": "$eni_#1", "destination": "1.4.0.1/32"}, "attributes": ["SAI_OUTBOUND_ROUTING_ENTRY_ATTR_ACTION", "SAI_OUTBOUND_ROUTING_ENTRY_ACTION_ROUTE_VNET", "SAI_OUTBOUND_ROUTING_ENTRY_ATTR_DST_VNET_ID", "$vnet_#eni1"]}, 
     {"name": "pa_validation_#eni1", "op": "create", "type": "SAI_OBJECT_TYPE_PA_VALIDATION_ENTRY", "key": {"switch_id": "$SWITCH_ID", "sip": "221.2.0.0", "vnet_id": "$vnet_#eni1"}, "attributes": ["SAI_PA_VALIDATION_ENTRY_ATTR_ACTION", "SAI_PA_VALIDATION_ENTRY_ACTION_PERMIT"]}, 
    ]
KrisNey-MSFT commented 3 weeks ago

DST MAC s/be in CA/PA mapping. Look at packet processing log. Overlay DST MAC = all 0's, yet bmv2 still forwards packet. Once exiting the pipeline, it s/be changed. Probably not in ENI address table. The reason this works: we don't know DST MAC at first. We have fake/dummy MAC (all 0's or other MACs), and this will work. If the MAC stays as all 0's, then we have a bug. Perhaps P4 code is not behaving as desired.
@mgheorghe will capture and check whether it works when exiting bmv2.