[OpenBMC] Python rspconfig get values not working

[Obihoernchen]

Hi,

the command rspconfig <node> ntpservers is not working with the python OpenBMC framework, but it works with perl:

[root@xcat ~]# rspconfig node1,node2  ntpservers -V
[xcat]: Running command in Python

[root@xcat ~]# chdef -t site clustersite openbmcperl="rspconfig"
1 object definitions have been created or modified.

[root@xcat ~]# rspconfig node1,node2 ntpservers -V
[xcat]: Running command in Perl
node1: [xcat]: BMC NTP Servers: 1.2.3.4,1.2.3.5
node2: [xcat]: BMC NTP Servers: 1.2.3.4,1.2.3.5

Setting ntpservers works fine though:

[root@xcat ~]# rspconfig node1 ntpservers=1.2.3.4,1.2.3.5
node1: BMC Setting NTPServers...
node1: BMC NTP Servers: 1.2.3.4,1.2.3.5
node1: Warning: time will not be synchronized until the host is powered off.

Version 2.14.6 (git commit 7c787c499ebca7a98d6fcdb3577f701ac16d7ab3, built Fri Mar 29 03:19:08 EDT 2019)
xCAT-openbmc-py-2.14.6-snap201903290319.noarch

[gurevichmark]

Can you check if there is an entry in /var/log/xcat/agent.log after rspconfig ntpservers in Python is ran, that says something like

 netinfo = obmc.get_netinfo()
  File "/opt/xcat/lib/python/agent/hwctl/openbmc_client.py", line xxx, in get_netinfo
    msg = "%s: Another valid ip %s found." % (node, v["Address"])
NameError: global name 'node' is not defined

[Obihoernchen]

No there is no such entry. Just a bunch of InsecureRequestWarning

[Obihoernchen]

While working with this I found more issues: 1) Setting multiple ntpservers has wrong API payload (broken in Python and Perl) When using rspconfig <node> server1,server2 the following is set (json API response):

      "NTPServers": [
        "X.X.X.X,X.X.X.Y"
      ],

But it should look like this (set with web interface):

      "NTPServers": [
        "X.X.X.X", 
        "X.X.X.Y"
      ],

I did a small quick fix for the python version:

}diff /tmp/openbmc_client.py /opt/xcat/lib/python/agent/hwctl/openbmc_client.py
796c796,800
<         payload = { "data": [servers] }
---
>         payload = { "data": servers.split(',') }

This works for one and multiple ntpservers.

2) Currently rspconfig sets NTPServers for the interface with the BMC IP only. In our case this is the eth0_16 interface. You can also see it configured for eth0_16 in the API response. But setting the NTP servers via rspconfig does not show them in the web interface and NTP is not working. Rebooting etc. makes no difference, NTP does not work when configured for interface eth0_16. We had to configure it for the default eth0 interface as well to make NTP work. Then it also shows up in the web interface. I don't know whether this is a firmware or xCAT bug. I will provide some logs via Slack.

[gurevichmark]

Opened related issue https://github.com/xcat2/xcat-core/issues/6230 while looking at this problem.

[gurevichmark]

@Obihoernchen Thank you for item 1 suggestion. For item 2 we need to setup a web interface for our lab machines to see how ntpservers set using rspconfig appear in the web panel.

[xuweibj]

Hi @Obihoernchen , is there other command has this problem? For example rspconfig hostname or other rspconfig commands.

[Obihoernchen]

Some are working, some are not:

# Perl
[root@xcat ~]# rspconfig  node1 ip -V
[xcat]: Running command in Perl
node1: [xcat]: BMC IP: 172.16.3.36
[root@xcat ~]# rspconfig  node1 hostname -V
[xcat]: Running command in Perl
node1: [xcat]: BMC Hostname: witherspoon

# Python
[root@xcat ~]# tabedit site
[root@xcat ~]# rspconfig  node1 ip -V
[xcat]: Running command in Python
[root@xcat ~]# rspconfig  node1 hostname -V
[xcat]: Running command in Python
[xcat]: node1: BMC Hostname: witherspoon

[Obihoernchen]

Oh and I'm sorry my answer regarding agent.log was wrong. There are indeed log entries. I didn't check the service nodes. Sorry!

/opt/xcat/lib/python/agent/hwctl/openbmc/openbmc_bmcconfig.py:13: RuntimeWarning: Patching more than once will result in the union of all True parameters being patched
  monkey.patch_all()
/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:769: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
Traceback (most recent call last):
  File "/opt/xcat/lib/python/agent/common/task.py", line 90, in run
    self._execute_in_parallel(op, *args, **kwargs)
  File "/opt/xcat/lib/python/agent/common/task.py", line 69, in _execute_in_parallel
    func(*args, node=node, nodeinfo=self.inventory[node], **kw)
  File "/opt/xcat/lib/python/agent/hwctl/openbmc/openbmc_bmcconfig.py", line 275, in get_attributes
    ntpservers=netinfo_dict.get('ntpservers', False), **kw)
  File "/opt/xcat/lib/python/agent/hwctl/openbmc/openbmc_bmcconfig.py", line 558, in _get_netinfo
    netinfodict['ip'].append("BMC IP"+addon_string+": %s" % attrs["ip"])
KeyError: 'ip'

I would use something like this to fix it:

# diff /opt/xcat/lib/python/agent/hwctl/openbmc/openbmc_bmcconfig.bak.py /opt/xcat/lib/python/agent/hwctl/openbmc/openbmc_bmcconfig.py
558,563c558,563
<             netinfodict['ip'].append("BMC IP"+addon_string+": %s" % attrs["ip"])
<             netinfodict['netmask'].append("BMC Netmask"+addon_string+": %s" % utils.mask_int2str(attrs["netmask"]))
<             netinfodict['gateway'].append("BMC Gateway"+addon_string+": %s (default: %s)" % (attrs["gateway"], defaultgateway))
<             netinfodict['vlan'].append("BMC VLAN ID"+addon_string+": %s" % attrs["vlanid"])
<             netinfodict['ipsrc'].append("BMC IP Source"+addon_string+": %s" % attrs["ipsrc"])
<             netinfodict['ntpservers'].append("BMC NTP Servers"+addon_string+": %s" % attrs["ntpservers"])
---
>             netinfodict['ip'].append("BMC IP"+addon_string+": %s" % attrs.get("ip", None))
>             netinfodict['netmask'].append("BMC Netmask"+addon_string+": %s" % utils.mask_int2str(attrs.get("netmask", 24)))
>             netinfodict['gateway'].append("BMC Gateway"+addon_string+": %s (default: %s)" % (attrs.get("gateway", None), defaultgateway))
>             netinfodict['vlan'].append("BMC VLAN ID"+addon_string+": %s" % attrs.get("vlanid", None))
>             netinfodict['ipsrc'].append("BMC IP Source"+addon_string+": %s" % attrs.get("ipsrc", None))
>             netinfodict['ntpservers'].append("BMC NTP Servers"+addon_string+": %s" % attrs.get("ntpservers", None))

[root@xcat ~]# rspconfig  node1 ip
node1: BMC IP for eth0_16: X.X.X.X
node1: BMC IP for eth0: None
[root@xcat ~]# rspconfig  node1 ntpservers
node1: BMC NTP Servers for eth0_16: X.X.X.Y
node1: BMC NTP Servers for eth0: None

[gurevichmark]

@Obihoernchen Thanks for the fix suggestion. That will fix your initial issue of rspconfig get not working in Python. But I think the second item of ntpservers setting not working and not properly displayed in Web interface is still there.

[Obihoernchen]

yes exactly!

[xuweibj]

Opened issue https://github.com/openbmc/openbmc/issues/3527 for asking help from firmware team.