Open kjacque opened 2 years ago
Can you post the verbose output? Or email it to me directly (my address is in my profile)
If you what dimms are there can you try the command for each of the individual dimms? For example
ipmctl show -dimm 0x0001
Alas, our datacenter powercycled the machine, and the DIMM came back up normally. I didn't manage to capture the full output beforehand.
I did save some output from the ipmctl start -diagnostic
command though, if that can help you track down the source of the segfault:
[root@wolf-143 ~]# ipmctl start -diagnostic
--Test = Quick
State = Warning
--SubTest = Manageability
State = Warning
Message.1 = The quick health check detected that PMem module 0x0101 is not manageable because firmware API version N/A is not supported.
--SubTest = Boot status
State = Ok
--SubTest = Health
State = Ok
--Test = Config
State = Failed
--SubTest = PMem module specs
State = Failed
Message.1 = The platform configuration check detected that PMem module with physical ID 0x0101 is present in the system but failed to initialize.
--SubTest = Duplicate PMem module
State = Ok
--SubTest = System Capability
State = Ok
--SubTest = Namespace LSA
State = Ok
--SubTest = PCD
State = Ok
--Test = Security
State = Ok
Message = The security check succeeded.
--SubTest = Encryption status
State = Ok
--SubTest = Inconsistency
State = Ok
--Test = FW
State = Ok
Message = The firmware consistency and settings check succeeded.
--SubTest = FW Consistency
State = Ok
--SubTest = Viral Policy
State = Ok
--SubTest = Threshold check
State = Ok
--SubTest = System Time
State = Ok
[root@wolf-143 ~]#
When I was poking around in dmidecode
it looked like the SMBIOS tables weren't showing much of anything for physical ID 0x101.
Thanks. Glad to hear that the dimm came back up in a good state.
We will try to figure out the cause of the seg fault.
Found the root cause, thanks @kjacque!
Can you collect the nlogs and provide them to @sscargal? They might provide additional information on why the PMem module failed.
ipmctl dump -destination out -dict <dict.txt> -dimm
. The dict.txt should be in the firmware image distributed to customers. If you have any questions, please talk with @sscargal.
@nolanhergert I've given up access to the DIMMs but I'll forward the information to our datacenter sysadmins in case they see something similar again. Thanks for the quick response. :)
@kjacque -- is there more we can do or can the issue be closed?
Okay to close.
Version: Intel(R) Optane(TM) Persistent Memory Command Line Interface Version 02.00.00.3885 OS: CentOS Linux release 8.4.2105
While trying to debug a problem with some faulty DIMMs, I see a segfault while running ipmctl show -dimm. It is always reproducible, in the state this machine is in anyway.
Running in verbose mode, I see the following errors immediately before the segfault:
Here's the call stack pulled from gdb: