entity-manager take up too many resources

[jack206]

CPU: 63% usr 35% sys 0% nic 0% idle 0% io 0% irq 0% sirq Load average: 5.90 5.09 4.25 2/102 2848 PID PPID USER STAT VSZ %VSZ %CPU COMMAND 1177 1 root R 10416 2% 55% /usr/bin/entity-manager 1182 1 root D 7808 2% 10% /usr/bin/fansensor 1180 1 root D 7432 2% 7% /usr/bin/adcsensor 1185 1 root D 7804 2% 6% /usr/bin/psusensor 1184 1 root S 6804 2% 3% /usr/bin/hwmontempsensor 2836 31946 root R 3116 1% 2% top 2210 1661 root S 3116 1% 1% top 343 340 messageb S 3696 1% 1% dbus-broker --log 4 --controller 9 --machine-id 734d44fa48f34a3a9a089754db3d802e --max-bytes 536870912 --max-fds 4096 --max-matches 16384 --audit 31929 1 root S 3752 1% 1% /usr/sbin/dropbear -i -r /etc/dropbear/dropbear_rsa_host_key -G priv-admin -B 973 1 root S 45060 11% 1% /usr/sbin/rsyslogd -n 171 1 root S 34072 8% 1% /lib/systemd/systemd-journald 426 1 root S 18040 4% 1% ipmid

Sep 27 08:37:09 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W_ xyz.openbmc_project.FruDevice Sep 27 08:37:09 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____0 xyz.openbmc_project.FruDevice Sep 27 08:37:09 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____1 xyz.openbmc_project.FruDevice Sep 27 08:37:09 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____2 xyz.openbmc_project.FruDevice Sep 27 08:37:11 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/BMC xyz.openbmc_project.FruDevice Sep 27 08:37:11 BMC entity-manager[1177]: probeDBus: Found probe match on /xyz/openbmc_project/FruDevice/BMC xyz.openbmc_project.FruDevice Sep 27 08:37:11 BMC entity-manager[1177]: probeDBus /xyz/openbmcproject/FruDevice/CRPS1600W xyz.openbmc_project.FruDevice Sep 27 08:37:11 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____0 xyz.openbmc_project.FruDevice Sep 27 08:37:11 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____1 xyz.openbmc_project.FruDevice Sep 27 08:37:11 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____2 xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/BMC xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W_ xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus: Found probe match on /xyz/openbmcproject/FruDevice/CRPS1600W xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____0 xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus: Found probe match on /xyz/openbmc_project/FruDevice/CRPS1600W____0 xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____1 xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus: Found probe match on /xyz/openbmc_project/FruDevice/CRPS1600W____1 xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____2 xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus: Found probe match on /xyz/openbmc_project/FruDevice/CRPS1600W____2 xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/BMC xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W_ xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____0 xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____1 xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____2 xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: run 604 Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/BMC xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmcproject/FruDevice/CRPS1600W xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____0 xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____1 xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____2 xyz.openbmc_project.FruDevice Sep 27 08:37:15 BMC entity-manager[1177]: ~PerformScan 621 Sep 27 08:37:16 BMC entity-manager[1177]: ~PerformScan 630 Sep 27 08:37:16 BMC entity-manager[1177]: findDbusObjects 205 Sep 27 08:37:16 BMC entity-manager[1177]: run 604 Sep 27 08:37:17 BMC entity-manager[1177]: getInterfaces 86 Sep 27 08:37:17 BMC entity-manager[1177]: getInterfaces 86 Sep 27 08:37:17 BMC entity-manager[1177]: getInterfaces 86 Sep 27 08:37:17 BMC entity-manager[1177]: getInterfaces 86 Sep 27 08:37:17 BMC entity-manager[1177]: getInterfaces 86 Sep 27 08:37:17 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/BMC xyz.openbmc_project.FruDevice Sep 27 08:37:17 BMC entity-manager[1177]: probeDBus: Found probe match on /xyz/openbmc_project/FruDevice/BMC xyz.openbmc_project.FruDevice Sep 27 08:37:17 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W_ xyz.openbmc_project.FruDevice Sep 27 08:37:17 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____0 xyz.openbmc_project.FruDevice Sep 27 08:37:17 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____1 xyz.openbmc_project.FruDevice Sep 27 08:37:17 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____2 xyz.openbmc_project.FruDevice Sep 27 08:37:18 BMC ipmid[426]: thresholdChanged: deassert Sep 27 08:37:20 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/BMC xyz.openbmc_project.FruDevice Sep 27 08:37:21 BMC entity-manager[1177]: probeDBus: Found probe match on /xyz/openbmc_project/FruDevice/BMC xyz.openbmc_project.FruDevice Sep 27 08:37:21 BMC entity-manager[1177]: probeDBus /xyz/openbmcproject/FruDevice/CRPS1600W xyz.openbmc_project.FruDevice Sep 27 08:37:21 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____0 xyz.openbmc_project.FruDevice Sep 27 08:37:21 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____1 xyz.openbmc_project.FruDevice Sep 27 08:37:21 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____2 xyz.openbmc_project.FruDevice Sep 27 08:37:24 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/BMC xyz.openbmc_project.FruDevice Sep 27 08:37:24 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W_ xyz.openbmc_project.FruDevice Sep 27 08:37:24 BMC entity-manager[1177]: probeDBus: Found probe match on /xyz/openbmcproject/FruDevice/CRPS1600W xyz.openbmc_project.FruDevice Sep 27 08:37:24 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____0 xyz.openbmc_project.FruDevice Sep 27 08:37:24 BMC entity-manager[1177]: probeDBus: Found probe match on /xyz/openbmc_project/FruDevice/CRPS1600W____0 xyz.openbmc_project.FruDevice Sep 27 08:37:24 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____1 xyz.openbmc_project.FruDevice Sep 27 08:37:24 BMC entity-manager[1177]: probeDBus: Found probe match on /xyz/openbmc_project/FruDevice/CRPS1600W____1 xyz.openbmc_project.FruDevice Sep 27 08:37:24 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____2 xyz.openbmc_project.FruDevice Sep 27 08:37:24 BMC entity-manager[1177]: probeDBus: Found probe match on /xyz/openbmc_project/FruDevice/CRPS1600W____2 xyz.openbmc_project.FruDevice Sep 27 08:37:24 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/BMC xyz.openbmc_project.FruDevice Sep 27 08:37:24 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W_ xyz.openbmc_project.FruDevice Sep 27 08:37:24 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____0 xyz.openbmc_project.FruDevice Sep 27 08:37:24 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____1 xyz.openbmc_project.FruDevice Sep 27 08:37:24 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____2 xyz.openbmc_project.FruDevice Sep 27 08:37:25 BMC entity-manager[1177]: run 604 Sep 27 08:37:25 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/BMC xyz.openbmc_project.FruDevice Sep 27 08:37:25 BMC entity-manager[1177]: probeDBus /xyz/openbmcproject/FruDevice/CRPS1600W xyz.openbmc_project.FruDevice Sep 27 08:37:25 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____0 xyz.openbmc_project.FruDevice Sep 27 08:37:25 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____1 xyz.openbmc_project.FruDevice Sep 27 08:37:25 BMC entity-manager[1177]: probeDBus /xyz/openbmc_project/FruDevice/CRPS1600W____2 xyz.openbmc_project.FruDevice Sep 27 08:37:25 BMC entity-manager[1177]: ~PerformScan 621

[jack206]

The background log keeps scrolling

[jack206]

root@BMC:~# time busctl tree xyz.openbmc_project.EntityManager | cat └─/xyz └─/xyz/openbmc_project ├─/xyz/openbmc_project/EntityManager └─/xyz/openbmc_project/inventory └─/xyz/openbmc_project/inventory/system ├─/xyz/openbmc_project/inventory/system/chassis │ └─/xyz/openbmc_project/inventory/system/chassis/BMC │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/BOARD_CURRENT2 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_P0V6_VTT │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_P1V05 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_P1V2_VDDQ │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_P1V2_VDDQ_V │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_P1V8 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_P1V8_AUX │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_P2V5_VPP │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_P3V3_AUX │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_P3V3_STBY │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_P5V_AUX │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_TEMP │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_V1P8_EMMC_O │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_V3P3_EMMC_O │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_VCCIN │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_VCC_ANA │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_VNN_NAC │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_VNN_PCH │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_XDPE_P1V05_V │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_XDPE_P1V8_V │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_XDPE_PCH_V │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_XDPE_VCCIN_V │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_XDPE_VCC_V │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/CPU_XDPE_VNN_V │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/FAN_OUTLET_TEMP0 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/FAN_OUTLET_TEMP1 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_TEMP │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_VDD12V │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_VDD12V_1 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_VDD1V0_FPGA │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_VDD1V2 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_VDD1V2_FPGA │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_VDD1V8_CLK │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_VDD1V8_FPGA │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_VDD3V3 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_VDD3V3_CLK │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_VDD3V3_STBY │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_VDD5V_VR │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_VDDO1V2 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_VDDO1V8 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MAC_VDD_CORE │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MGMT_INLET_TEMP0 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/MGMT_INLET_TEMP1 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN1_1 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN1_2 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN2_1 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN2_2 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN3_1 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN3_2 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN4_1 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN4_2 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN5_1 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN5_2 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN6_1 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN6_2 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN7_1 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN7_2 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN8_1 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/ODM_FAN8_2 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/PSU1 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/PSU2 │ ├─/xyz/openbmc_project/inventory/system/chassis/BMC/PSU3 │ └─/xyz/openbmc_project/inventory/system/chassis/BMC/PSU4 └─/xyz/openbmc_project/inventory/system/powersupply ├─/xyz/openbmc_project/inventory/system/powersupply/CRPS1600W_1 ├─/xyz/openbmc_project/inventory/system/powersupply/CRPS1600W_2 ├─/xyz/openbmc_project/inventory/system/powersupply/CRPS1600W_3 └─/xyz/openbmc_project/inventory/system/powersupply/CRPS1600W_4

real 3m9.069s user 0m0.490s sys 0m0.123s

[williamspatrick]

What hardware are you running this on? Is the EM JSON all upstreamed or do you have your own JSON you are loading? Are you able to share it if it isn’t upstreamed?

[jack206]

AST2500,the EM file is: bmc_chassis.json crps1600w_psu.json fan12k8080_fan.json

[jack206]

oem_xxx type You can interpret it as xxx type

[edtanous]

I see a number of logs, but given you're using a non upstreamed system on a fork, it's nearly impossible for anyone to debug this for you. One possibility is that something on your system is repeatedly calling the DBus ReScan() method, which is causing EM to rescan over and over. It's difficult to make any more guesses without being able to see the complete code, and being based on a fork.

[williamspatrick]

AST2500,the EM file is: bmc_chassis.json crps1600w_psu.json fan12k8080_fan.json

I don’t see anything obviously wrong here that would have itself cause performance issues as you’ve described. I will say that the probe statement in the chassis object is far too unspecified. Just looking for “BMC” is a lot less than desirable; certainly this never could be upstreamed in its current state. I would suggest you get your backplane eeprom to have more specific details in it.

[jack206]

I am based on that commit： commit b0c7bd18c20983a565c21e1921d647132329f789 (tag: devtool-base, master) Author: Zev Weiss zev@bewilderbeest.net Date: Tue Apr 19 17:58:01 2022 -0700

romed8hm3: add VDDCR_CPU warning & critical thresholds

These had been omitted initially while we were sorting out some
confusion with the VRM that provides this supply rail, but I forgot to
go back and add them after that was resolved.

Tested: warning & critical thresholds appear as expected on dbus and in
webui-vue on romed8hm3.

Signed-off-by: Zev Weiss <zev@bewilderbeest.net>
Change-Id: I605c7813f70b5004f50be831eea4746e48a4a5bd