Closed openhpi2 closed 8 years ago
Rafael,
Could you please give more information like what plugin was used, was it a make install or rpm install etc. Unable to reproduce it.
Thanks Mohan
Original comment by: dr_mohan
Hi,
I'm afraid I don't have much useful information. My openhpi.conf file contains only OPENHPI_LOG_ON_SEV = "DEBUG" all the rest is commented out.
I compiled and ran openhpid from trunk, without installing. This is the log I get on the deamon when I run hpi_shell: openhpid: INFO: server.cpp:125: Got connection from 127.0.0.1 port 43392 openhpid: DBG: server.cpp:194: ### Spawning thread to handle connection. ### openhpid: DBG: server.cpp:227: 0x15a9d90 Servicing connection. openhpid: DBG: server.cpp:233: ### service_thread, thrdid [0x15a9d90] ### openhpid: DBG: server.cpp:341: 0x15a9d90 Processing RPC request 1. openhpid: DBG: server.cpp:367: OpenHPID calling the saHpiSessionOpen openhpid: DBG: server.cpp:2112: 0x15a9d90 Return code = 0 openhpid: DBG: server.cpp:341: 0x15a9d90 Processing RPC request 4. openhpid: DBG: server.cpp:2112: 0x15a9d90 Return code = 0 openhpid: DBG: server.cpp:341: 0x15a9d90 Processing RPC request 3. openhpid: DBG: threaded.c:234: Going to wait for discovery thread to loop once. openhpid: DBG: threaded.c:49: Discovery: Iteration. openhpid: CRIT: plugin.c:470: Warning - no handlers openhpid: DBG: threaded.c:52: Got error on threaded discovery return. openhpid: DBG: threaded.c:61: Discovery: Going to sleep. openhpid: DBG: threaded.c:237: Got signal from discovery thread being done. Giving lock back. openhpid: DBG: server.cpp:2112: 0x15a9d90 Return code = 0 openhpid: DBG: server.cpp:300: 0x15a9d90 Connection closed. openhpid: DBG: threaded.c:92: Event harvesting: Iteration. openhpid: CRIT: plugin.c:470: Warning - no handlers openhpid: CRIT: threaded.c:95: Error on harvest of events.
Original comment by: rdossant
So here is what I do to reproduce the problem:
Original comment by: rdossant
Hi Rafael,
I tried to recreate this issue and was able to recreate on one of our machine which has Debian OS with sid distribution.
praveen@debian8:~/3.2.x$ cat /etc/issue Debian GNU/Linux stretch/sid \n \l
Steps followed:
praveen@debian8:~/3.2.x$ hpi_shell hpi_shell - This program came with OpenHPI 3.7.0 SAF HPI Version B.03.02
Attempt to unlock mutex that was not locked Aborted (core dumped)
When I tried the same on other Debian machine which has different distribution such as squeeze, this issue does not exist over there and it works fine without any issue.
Could you pelase let us know osme more details of the machine where you have observed this issue.
Any help is appreciated.
Thanks! Praveen
Original comment by: openhpi2
Praveen,
On 13 January 2016 at 13:50, Praveen erpraveenk@users.sf.net wrote:
Could you pelase let us know osme more details of the machine where you have observed this issue.
- Machine details such as OS version, distribution, libglib etc.
- What is scenario to reproduce in your case, as I am able to reproduce only when I issue “hpi_shell” when discovery is in progress.
- Linux 4.2.8-300.fc23.x86_64, Fedora 23, glibc-2.22-7, glib2-2.46.2-1
- I can get hpi_shell to fail anytime after I ran the openhpi daemon. Here is an example of output from the daemon when reproducing the problem:
$: sudo ./openhpid/openhpid -c /etc/openhpi/openhpi.conf -v -n openhpid: INFO: openhpid-posix.cpp:420: Starting OpenHPI 3.7.0. openhpid: INFO: openhpid-posix.cpp:422: OPENHPI_CONF = /etc/openhpi/openhpi.conf. openhpid: INFO: openhpid-posix.cpp:427: OPENHPI_DAEMON_PORT = 4743. openhpid: INFO: openhpid-posix.cpp:430: Enabled IP versions: IPv4. openhpid: INFO: openhpid-posix.cpp:431: Max threads: -1. openhpid: INFO: openhpid-posix.cpp:432: Socket timeout(sec): 0. openhpid: DBG: event.c:50: Setting up event processing queue. openhpid: DBG: event.c:53: Set up processing queue. openhpid: INFO: init.c:90: Loading config file /etc/openhpi/openhpi.conf. openhpid: DBG: conf.c:678: Done processing conf file. Parse errors: 0 openhpid: INFO: init.c:111: Initializing UID. utils: INFO: uid_utils.c:164: Using UID Map file /var/lib/openhpi/uid_map. openhpid: INFO: init.c:128: Creating default domain. openhpid: INFO: init.c:131: Auto-Insert Timeout is 0 nsec. openhpid: INFO: init.c:135: Auto-Insert Timeout is READ-ONLY. openhpid: DBG: threaded.c:153: Starting discovery thread. openhpid: DBG: threaded.c:159: Starting event threads. openhpid: INFO: init.c:174: OpenHPI has been initialized. openhpid: WARN: init.c:181: Warning: No handler definitions found in config file. Check configuration file and previous messages openhpid: DBG: threaded.c:45: Begin discovery. openhpid: DBG: threaded.c:49: Discovery: Iteration. openhpid: DBG: threaded.c:125: Begin event processing. openhpid: CRIT: plugin.c:470: Warning - no handlers openhpid: DBG: threaded.c:52: Got error on threaded discovery return. openhpid: DBG: threaded.c:61: Discovery: Going to sleep. openhpid: DBG: threaded.c:88: Begin event harvesting. openhpid: DBG: threaded.c:92: Event harvesting: Iteration. openhpid: CRIT: plugin.c:470: Warning - no handlers openhpid: CRIT: threaded.c:95: Error on harvest of events. openhpid: DBG: threaded.c:101: Event harvesting: Going to sleep. openhpid: INFO: server.cpp:125: Got connection from 127.0.0.1 port 36248 openhpid: DBG: server.cpp:194: ### Spawning thread to handle connection. ### openhpid: DBG: server.cpp:227: 0xa88cf0 Servicing connection. openhpid: DBG: server.cpp:233: ### service_thread, thrdid [0xa88cf0] ### openhpid: DBG: server.cpp:341: 0xa88cf0 Processing RPC request 1. openhpid: DBG: server.cpp:367: OpenHPID calling the saHpiSessionOpen openhpid: DBG: server.cpp:2112: 0xa88cf0 Return code = 0 openhpid: DBG: server.cpp:341: 0xa88cf0 Processing RPC request 4. openhpid: DBG: server.cpp:2112: 0xa88cf0 Return code = 0 openhpid: DBG: server.cpp:341: 0xa88cf0 Processing RPC request 3. openhpid: DBG: threaded.c:234: Going to wait for discovery thread to loop once. openhpid: DBG: threaded.c:49: Discovery: Iteration. openhpid: CRIT: plugin.c:470: Warning - no handlers openhpid: DBG: threaded.c:52: Got error on threaded discovery return. openhpid: DBG: threaded.c:61: Discovery: Going to sleep. openhpid: DBG: threaded.c:237: Got signal from discovery thread being done. Giving lock back. openhpid: DBG: server.cpp:2112: 0xa88cf0 Return code = 0 openhpid: DBG: server.cpp:300: 0xa88cf0 Connection closed. openhpid: DBG: threaded.c:92: Event harvesting: Iteration. openhpid: CRIT: plugin.c:470: Warning - no handlers openhpid: CRIT: threaded.c:95: Error on harvest of events. openhpid: DBG: threaded.c:101: Event harvesting: Going to sleep. ^Copenhpid: DBG: event.c:460: Processing event for domain 0 openhpid: DBG: server.cpp:200: Server socket closed. openhpid: DBG: event.c:285: Added event to EL openhpid: DBG: server.cpp:203: All connection threads are terminated. openhpid: DBG: event.c:295: Got session list for domain 0 openhpid: DBG: event.c:302: No sessions open for event's domain 0. Dropping hpi_event openhpid: DBG: threaded.c:135: Done with event processing. openhpid: DBG: threaded.c:116: Done with event harvesting. openhpid: DBG: threaded.c:78: Done with discovery. openhpid: DBG: event.c:65: Processing queue is disposed. openhpid: INFO: init.c:221: OpenHPI has been finalized.
Rafael Fonseca
Original comment by: rdossant
Thanks Rafael for your quick response.
From the above mentioned log, I can see that there are no handler registered when the plug-in has started. Hence, once you start the daemon and after that are you able to get the proper output for other client programs such as hpiinv, hpitree -a, hpisensor etc. Is the behaviour same as hpi_shell or is it different when you execute other client programs.
Thanks! Praveen
Original comment by: openhpi2
Praveen,
On 14 January 2016 at 14:18, Praveen erpraveenk@users.sf.net wrote:
Thanks Rafael for your quick response.
From the above mentioned log, I can see that there are no handler registered when the plug-in has started. Hence, once you start the daemon and after that are you able to get the proper output for other client programs such as hpiinv, hpitree -a, hpisensor etc. Is the behaviour same as hpi_shell or is it different when you execute other client programs.
I tried all the binaries in the clients/ directory, and I could not reproduce the error with any of them (they all executed normally, without aborting). It only happens with hpi_shell.
Att. Rafael Fonseca
Original comment by: rdossant
Rafael,
Could you please share me your config file details as well as client output for hpitree -a and hpiinv.
Thanks! Praveen
Original comment by: openhpi2
$: ./clients/hpitree -a ~/prjs/openhpi-code/openhpi/trunk/clients/.libs/lt-hpitree - This program came with OpenHPI 3.7.0 SAF HPI Version B.03.02
Discovery done Entity Path: {396,0}{1571218171,32742}{1573251072,32742}{396,0}{1547608220,32742}{1573250256,32742}{1569016808,32742}{1571218292,32742}{1556963232,32742}{1569022166,32742}{BACK_PANEL_BOARD,0}{1177260628,32766}{1177260632,32766}{1573398656,32742}{1573398576,32742}{1556963232,32742}
EntryId: 1556915080
ResourceId: 32742
Resource Information:
Resource Revision: 39
Specific Version: 3
Product ID: 10368
Firmware Major Revision: 200
Firmware Minor Revision: 93
Aux Firmware Revision: 230
GUID: 7f000001-eca6-5de6-7f00-002703000000
Entity Path: {396,0}{1571218171,32742}{1573251072,32742}{396,0}{1547608220,32742}{1573250256,32742}{1569016808,32742}{1571218292,32742}{1556963232,32742}{1569022166,32742}{BACK_PANEL_BOARD,0}{1177260628,32766}{1177260632,32766}{1573398656,32742}{1573398576,32742}{1556963232,32742}
Capabilities:
AGGREGATE_STATUS | ANNUNCIATOR | FRU | RESET | RESOURCE | WATCHDOG
HotSwap Capabilities: None
Resource Severity: (null)
ResourceFailed: FALSE
ResourceTag:
Data Type: (null)
Language: UNDEF
Data Length: 128
Data: (È]æ
$: ./clients/hpiinv ~/prjs/openhpi-code/openhpi/trunk/clients/.libs/lt-hpiinv - This program came with OpenHPI 3.7.0 SAF HPI Version B.03.02
RptEntryGet: rid=0 rv = -1011 RptEntryGet: rid=0 rv = -1011 RptEntryGet: rid=0 rv = -1011 RptEntryGet: rid=0 rv = -1011 RptEntryGet: rid=0 rv = -1011 RptEntryGet: rid=0 rv = -1011 RptEntryGet: rid=0 rv = -1011
Original comment by: rdossant
Thanks Rafael for sharing the client output and configuration file.
I went through them and have few more questions, could you please address them.
Thanks! PraveeN
Original comment by: openhpi2
On 21 January 2016 at 17:45, Praveen erpraveenk@users.sf.net wrote:
Thanks Rafael for sharing the client output and configuration file.
I went through them and have few more questions, could you please address them.
1.
I could not see any handler registered as in your configuration file I cannot see any plug-in configured. In order to support any plug-in, it needs to be installed and configured. 2.
I used your configuration file in my test environment but I could not see any code dump or error mentioned with hpi_shell client program. For hpitree –a and hpiinv results are similar to yours. 3.
As you have mentioned that you are getting this issue reproduced on Fedora 23, could you please try this on RHEL6/7 and let me know the outcome. 4.
Could you please share me the command output of ‘rpm -qa | grep openhpi’ in order to find the packages loaded.
- Tried again enabling the simulator plugin. Config and logs attached. I got the same result.
- hmm, interesting. At first I thought it could be related to the gcc flags used by Fedora to compile packages (like hardened_build, -fPIC, ...). But I also tried compiling upstream openhpi with only the following configure options ./configure --prefix=/tmp/ --sysconfdir=/etc/ --with-varpath=/var/lib/openhpi and the result is the same.
- I could reproduce it in RHEL7.2.
- Sure Fedora 23: openhpi-libs-3.6.1-1.fc23.x86_64 openhpi-debuginfo-3.4.0-1.fc22.x86_64 openhpi-3.6.1-1.fc23.x86_64
RHEL7.2: openhpi-libs-3.4.0-2.el7.x86_64 openpi-3.4.0-2.el7.x86_64
Rafael Fonseca
Original comment by: rdossant
Hi Rafael,
I am unable to reproduce this issue exactly in my lab environment. If possible, could you please give a try with OpenHPI 3.6.1 on RHEL7 and let me know the result.
Thanks! PraveeN
Original comment by: openhpi2
Praveen,
On 1 February 2016 at 16:48, Praveen erpraveenk@users.sf.net wrote:
Hi Rafael,
I am unable to reproduce this issue exactly in my lab environment. If possible, could you please give a try with OpenHPI 3.6.1 on RHEL7 and let me know the result.
I can still reproduce it with OpenHPI 3.6.1 in RHEL7.
Rafael Fonseca
Original comment by: rdossant
Hi Rafael,
Though we have tried to re-create this issue on multiple operating systems, we are not able to re-create this issue. I have followed most of steps/configuration provided from your end, still I am not able to find any core dump or the error message mentioned from your end.
At latest, I have tried it to reproduce on RHEL7, and here I am mentioning the details of the steps/configuration followed.
Discovery done Enter a command or "help" for list of commands Available commands are:
addcfg ann clearevtlog ctrl dat debug dimi domain domaininfo drt dscv echo entinstr entres event evtlogtime evtlogreset evtlogstate exec fumi help history hs inv lsent lsres lsensor more newdomain parmctrl power quit rdr rdrupdatecounter reopen removefailed reset rpt run sen settag setsever settimeevtlog showevtlog showinv showrdr showrpt ver wdtget wdtreset wdtset ?
hpi_shell> inv NO rpts! Command failed: inv hpi_shell> q quit [root@localhost ~]#
[root@localhost ~]# rpm -qa | grep openhpi openhpi-dynamic_simulator-3.6.0-1.x86_64 openhpi-debuginfo-3.6.0-1.x86_64 openhpi-bladecenter-3.6.0-1.x86_64 openhpi-slave-3.6.0-1.x86_64 openhpi-clients-3.6.0-1.x86_64 openhpi-ipmidirect-3.6.0-1.x86_64 openhpi-test_agent-3.6.0-1.x86_64 openhpi-devel-3.6.0-1.x86_64 openhpi-oa_soap-3.6.0-1.x86_64 openhpi-watchdog-3.6.0-1.x86_64 openhpi-3.6.0-1.x86_64 openhpi-simulator-3.6.0-1.x86_64 openhpi-ilo2_ribcl-3.6.0-1.x86_64 [root@localhost ~]# lsb_release -a LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch Distributor ID: RedHatEnterpriseServer Description: Red Hat Enterprise Linux Server release 7.0 Alpha3 (Maipo) Release: 7.0 Codename: Maipo [root@localhost ~]#
Here, I do not get any error message or the core dump as you are getting consistently across the different operating systems. Here I would like to know few points from you:
Attached:
Thanks! PraveeN
Original comment by: openhpi2
On 24 February 2016 at 14:50, Praveen erpraveenk@users.sf.net wrote:
Hi Rafael,
[root@localhost ~]# lsb_release -a LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch Distributor ID: RedHatEnterpriseServer Description: Red Hat Enterprise Linux Server release 7.0 Alpha3 (Maipo) Release: 7.0 Codename: Maipo
Hmm, RHEL-7.0 Alpha3 is really old (~4 years). What gcc version and glib2 are you using? Do you have redhat-rpm-config installed?
I can also reproduce this in my Fedora 23. Would it be possible for you to try to reproduce it in Fedora?
[root@localhost ~]#
Here, I do not get any error message or the core dump as you are getting consistently across the different operating systems. Here I would like to know few points from you:
- Am I missing some steps in order to see this issue.
Nope. I did the exact same steps on RHEL-7.2 and got the mutex output.
- How different is your approach from mine, in which you are able to recreate this issue.
No difference apart from the OS used (consequently we might be using different gcc/glib2 versions).
- Are there any additional compilation packages or gcc flags enables on your system?
I'm attaching the config.log file where you can check the flags used.
Rafael Fonseca
Original comment by: rdossant
Original comment by: openhpi2
Fixed in trunk revision 7667.
Original comment by: openhpi2
Original comment by: openhpi2
./openhpid/openhpid -c /etc/openhpi/openhpi.conf
$: ./hpi_shell/hpi_shell openhpi-code/openhpi/trunk/hpi_shell/.libs/lt-hpi_shell - This program came with OpenHPI 3.7.0 SAF HPI Version B.03.02
Attempt to unlock mutex that was not locked
Discovery done Aborted (core dumped)
I might be doing something wrong, because under 'strace' the program works correctly.
----------- GDB output ---------- Attempt to unlock mutex that was not locked [New Thread 0x7ffff5fb3700 (LWP 26335)]
--------- GDB --------
Reported by: rdossant