search

ibm-openbmc / dev

Product Development Project Mgmt and Tracking
16 stars 2 forks source link

SNMP errors in the Journal #3599

Closed mzipse closed 1 year ago

mzipse commented 2 years ago

Internal number SW553815

We are seeing failures in the BMC Journal. Fortunately everything is still work. Would like the IPS team to investigate.

Here are the details I have: Test Procedure:

  1. Load the latest driver 1020.2226.20220630a
  2. Reset BMC or post update, check for errors

Noticed in journals there is a bmcweb trying to access over 443

Jul 01 04:55:46 rainjmt02bmc systemd[1]: Starting Wait for /xyz/openbmc_project/network/snmp... Jul 01 04:55:49 rainjmt02bmc systemd[1]: Finished Wait for /xyz/openbmc_project/network/snmp. Jul 01 04:56:26 rainjmt02bmc bmcweb[1568]: (2022-07-01 04:56:26) [ERROR "event_service_manager.hpp":403] HttpClient connection is null. Create a conn for id:snmp1 destination: 9.41.166.76:443 Jul 01 04:56:41 rainjmt02bmc bmcweb[1568]: (2022-07-01 04:56:41) [ERROR "http_client.hpp":412] Maximum number of retries reached for Subscriber:snmp1 Jul 01 04:56:41 rainjmt02bmc bmcweb[1568]: (2022-07-01 04:56:41) [ERROR "http_client.hpp":425] TerminateAfterRetries is set. retryCount: 3 .Subscriber: snmp1

Had a discussion with Sunitha on this for "SNMPv2c" protocol type subscriptions, there should not be an http client object created. This was in place earlier... seems like broken now

"subscriptions":[{"Context":"","DeliveryRetryPolicy":"TerminateAfterRetries","Destination":"snmp://9.41.166.76", "EventFormatType":"Event","HttpHeaders":{},"Id":"snmp1","MessageIds":[],"MetricReportDefinitions":[], "Protocol":"SNMPv2c","RegistryPrefixes":[],"ResourceTypes":[],"SubscriptionType":"SNMPTrap"}], "system_uuid":"19b06751-28f3-43b6-b97f-4f103d0ad1b9","timeout":1800}

this needs a change in the bmcweb. This will not cause any functionality errors. But may fill the journal with traces like this, and its actually not needed to create a http client for snmp subscription and this is related to the IPS code - snmp trap subscription

ChicagoDuan commented 2 years ago

Hi @mzipse I have solved this bug and pushed a PR to ibm-openbmc/bmcweb: https://github.com/ibm-openbmc/bmcweb/pull/413 I have tested it on the rainier machine(10.250.3.20), and the error logs will not appear. Please review and test this PR. if it passes, I will push it to upstream.

mzipse commented 1 year ago

We've pulled the fix into our downstream builds. I'll let the testers verify.

gtmills commented 1 year ago

This was done here: https://github.com/ibm-openbmc/bmcweb/pull/413 Closing. Please reopen if still an issue