sonic-net / sonic-sairedis

SAI object interface to Redis database, as used in the SONiC project
Other
56 stars 273 forks source link

VXLAN tunnel bogus error `unexpected type: SAI_OBJECT_TYPE_TUNNEL` #1467

Closed bradh352 closed 2 days ago

bradh352 commented 2 days ago

When trying to debug VXLAN tunnels it is bad to have errors output in the logs that are false positives. This can lead users to go down a rabbit hole when the error has nothing to do with the issue at hand.

Here is the error sequence:

Aug 16 03:14:19.373668 Odin-ec58a ERR swss#orchagent: :- meta_sai_on_port_state_change_single: data.port_id oid:0x2a0000000009bf has unexpected type: SAI_OBJECT_TYPE_TUNNEL, expected PORT, BRIDGE_PORT or LAG
Aug 16 03:14:19.375495 Odin-ec58a NOTICE swss#orchagent: :- doTask: Get port state change notification id:2a0000000009bf status:1
Aug 16 03:14:19.375495 Odin-ec58a ERR swss#orchagent: :- doTask: Failed to get port object for port id 0x2a0000000009bf

In this case, as reported in https://github.com/sonic-net/sonic-buildimage/issues/10004 the fix is to simply add the SAI_OBJECT_TYPE_TUNNEL to the switch statement.

This is a trivial correction for an incorrect log entry. It doesn't appear to alter behavior in any way.

It has been confirmed via my own Dell S5248F switches to resolve this false-positive error message.

Signed-off-by: Brad House (@bradh352)

kcudnik commented 2 days ago

tunnel object is not expected there: https://github.com/opencomputeproject/SAI/blob/master/inc/saiport.h#L137 according to latest SAI on master only 3 objects are expected at that point, if tunnel is als oexpected teher, please bring that issue on SAI comunity meeting and post PR on SAI repo to update that metadata

bradh352 commented 2 days ago

@kcudnik

I don't intend on adding a new metadata port type to track by any means. I was just trying to silence that warning as it inhibits the ability of someone attempting to debug VXLAN issues. I guess I didn't fully evaluate what that warning was about.

So it sounds like the better thing to do is to find where that event originates and filter out SAI_OBJECT_TYPE_TUNNEL so it never tries to send a port metadata update using that object type.

Any idea off the top of your head where to do that? I've only been playing with SONiC for about 2 weeks, so I'm still quite green.

kcudnik commented 2 days ago

if onject type tunnel is send in notification and by current SAI headers this is not allowed, then this seems like vendor bug, otherwise SAI metadat should be updated, but not in this PR, im meking generic fix for this and i will post it shortly

kcudnik commented 2 days ago

if this iss issue for you you probably can post new issue on SAI https://github.com/opencomputeproject/SAI/issues and explain the origin

bradh352 commented 2 days ago

call chain appears to be Sai::handle_notification() -> RedisRemoteSaiInterface::syncProcessNotification() -> NotificationPortStateChange::processMetadata() -> Meta::meta_sai_on_port_state_change() -> Meta::meta_sai_on_port_state_change_single()

Sai::apiInitialize() makes a context which calls this notification callback.

The chain is a little harder to follow upstream from there. I'm assuming its being called from a different project, maybe swss->orchagent->portsarch ?

bradh352 commented 2 days ago

ah, i see you said you'll make a generic fix for that. Thanks.

And I also see you say its probably a vendor bug, so I guess that's bcm sai sending something it shouldn't is what you're saying.

kcudnik commented 2 days ago

maybe tunnel is also valid for that notification and metadata was not updated, that i dont know

fixed this part to use metadata: https://github.com/sonic-net/sonic-sairedis/pull/1469

kcudnik commented 2 days ago

will not merge this becaue of above fix