Dell PowerVault ME4: special agent crashes after latest firmware update

[mbunkus]

Hey,

today I've unfortunately updated our ME4's firmware to the latest version (see below). Now the special agent crashes. Here's its output.

The interesting thing is sometimes the check succeeds, but only very rarely.

I've tried restarting the management controllers, I've also tried deleting & recreating the monitoring user (with the same parameters as before, of course).

Both controllers are running the same version. The check is run against both management controllers, and for both the special agent fails this way.

The web interface seems to work fine; I can log in, configure stuff, reboot management controllers — all normal, no aborts, no errors. Monitoring via SNMP still works fine, too.

Any ideas?

ME4 MKP version: 2.4

Firmware details:

• Bundle version: GT280R010-01
• Storage controller code version: GTS280R10-26
• Storage controller loader code version: 29.015
• Management controller code version: GXM280R013-28
• Management controller loader code version: 6.29.27348x
• Expander controller code version: 52A0
• CPLD code version: 3.9

[Yogibaer75]

It say's that there are a wrong session key. The debug is not so easy here i think. One option is to insert a small debug line after line 53 with

print(r.json()) to get the raw response after the login.

[mbunkus]

Here's the JSON reply of the authentication call:

{'status': [{'object-name': 'status', 'meta': '/meta/status', 'response-type': 'Error', 'response-type-numeric': 1, 'response': 'Authentication Unsuccessful', 'return-code': 2, 'component-id': '', 'time-stamp': '2022-07-28 12:25:26'
, 'time-stamp-numeric': 1659011126}]}

I've tried both with the pre-existing monitoring user as well as my own personal user. In both cases I've copy-pasted both the username & the password from a text file to both the special agent command line and the web interface. The former produces the error shown, the web interface login works just fine — so I'm sure the credentials are correct.

I've been digging a bit deeper, looking for the official documentation for the ME4 REST API. I've found this page which states:

HTTPS authentication using an SHA256 hash to return a session key that is sent for each request. The session key is valid has a 30-minute inactivity timeout. Use of SHA256 is now recommended instead of MD5, which is deprecated. To log in to the HTTPS API, the username and password must be joined with an underscore as a separator (username_password). The username and password is then sent through an SHA256 hash. The SHA256 hash is represented in lower case hexadecimal format. This string is appended to the login function for the API, https://IP-address/api/login/hash. For example: …

The special agent still uses MD5 hashes, though. Next I tried simply swapping hashlib.md5(…) with hashlib.sha256(…), and lo and behold: it works again!

I have two other ME4s I'm monitoring using this special agent with slightly older firmware levels. I'll try using SHA-256 with those, too, and report back.

[mbunkus]

Management controller firmware versions GXM280R004-01 & GXM280R013-07 work fine with SHA-256, too.

[Yogibaer75]

If the older firmware works also with the sha256 setting - i will change the this to the new behavior

[mbunkus]

Great, thanks! I cannot tell you how far back SHA-256 is supported, though. I only have access to the three management controller firmware versions that I posted about.

[Yogibaer75]

Fixed with Commit

[mbunkus]

Thank you very much!