Closed thilles closed 5 years ago
Suspect It's the pymqi version and specifically the python2/3 compatibility. I'm busy converting to cater for python 3, pymqi 1.9+ and splunk 8.
Can u please advise on the following Splunk version Pymqi version. MQ client version
As a quick fix u could go back to pymqi 1.5. But I may have a patch out for this in the next few days.
Sure, here you go: Splunk 7.2.1 Pymqi 1.9.2 MQ client 9.1.3.0
I'll try pymqi 1.5 in the meanwhile
Still an error using pymqi 1.5.4, but now it's morphed somewhat:
ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mq_ta/bin/mqinput.py" Stopping. Exception in QueuePoller: MQCD wrong size. Given: 1984, expected 1992 mqinput_stanza:mqinput://Test1_mq_queue
Both input types throw the same error. This is also the same error I got when I, with version 1.9.2, casted the mq_user_name and mq_password to strings in mqinput.py and mqchs.py.
Indeed. My sincerest apologies. I'm busy resolving these compatibility issues as we speak. That particular one is due to the MQCD modifications in the later MQ versions.
This sucks. I should not have left this conversion for so late.
If you can wait a few days great. Otherwise - you need to downgrade all your MQ client software which is not something I would suggest.
Dont stress though. This plugin runs in a couple of production environments reading events from MQ - so this conversion is top priority.
I have made progress. I should have a new version available on Monday.
OK. Much better!
You can test with the latest code on the python2-to-python3-conversion branch if you like. See here: https://github.com/jjjw420/splunk-ibm-websphere-mq-add-on/tree/python2-to-python3-conversion
PLEASE NOTE: The current version of pymqi does not support MQ 9.1.3 yet (9.1.3 added an additional field to the MQCD structure which is the reason for your last error). However - you can find a branch that adds support for MQ 9.1.3 here: https://github.com/SeyfSV/pymqi/tree/mqcd-913/code/pymqi
Also note that Splunk V8 now has the "ctypes" library installed by default. So you no longer have to find a Splunk Python compatible version of "ctypes" and "_ctypes.so". Just the PyMQI library is required with Splunk V8 which makes the installation much simpler.
I am currently testing this build on:
If all is well I will be merging this back into master soonest.
On request I merged the latest changes for python3 and pymqi compatibility into master.
Please try it and let me know. Remember that if you use MQC 9.1.3 that you will need the branch from https://github.com/SeyfSV/pymqi/tree/mqcd-913/code/pymqi
Thanks for the fast work and quick responses!
Installed the mqcd-913
brach of pymqi and the latest master version of mq_ta, but still experiencing the same error ChannelStatusPoller: Both user and password must be instances of str or bytes mqchs_stanza:mqchs://Test2_channel_status
However, by casting the mq_user_name
and mq_password
to string in mqinput.py and mqchs.py the error disappeared and the input connected. Still get an error MQ Exception occurred: MQI Error. Comp: 2, Reason 2538: FAILED: MQRC_HOST_NOT_AVAILABLE
but that's more our problem I believe.
That was my fault. I only migrated the last of the queue managers which have authentication now. I fixed the user name and password unicode strings. You can pull the latest code. I'm going to check all the strings again.
MQRC 2538 only really happens for a few reasons - the most common being: 1.) You have the wrong host name, ip or the dns lookup fails. 2.) you have the wrong port. Please do me favour and actualy enter the port - even if its the 1414 default. I will verify this again too.
So check your connection parameters. However - We can easily debug the mqrc 2538 and your connection to MQ.
I normally use the "amqscnxc" test program which is installed with the MQ samples but you could also use "MQ Explorer".
eg.
/opt/mqm/samp/bin/amqscnxc -x 'myhostnameorip(listener port)' -c <server conn channel> [-u <username>] <queue manager name>
So given Queue Manager "LDB0" at "192.168.0.127" on port 1414 and using the "LDB0.CLIENT" server connection channel with no authentication - a good connection will look like this:
$ /opt/mqm/samp/bin/amqscnxc -x '192.168.0.127(1414)' -c LDB0.CLIENT LDB0
Sample AMQSCNXC start
Connecting to queue manager LDB0
using the server connection channel LDB0.CLIENT
on connection name 192.168.0.127(1414).
Connection established to queue manager LDB0
Sample AMQSCNXC end
If I use the wrong port or host(see the MQRC 2538):
$ /opt/mqm/samp/bin/amqscnxc -x '192.168.0.127(1417)' -c LDB0.CLIENT LDB0
Sample AMQSCNXC start
Connecting to queue manager LDB0
using the server connection channel LDB0.CLIENT
on connection name 192.168.0.127(1417).
MQCONNX ended with reason code 2538
With authentication it looks like this - given queue manager "LDB1" at "10.182.22.23(1415)" using the "LDB1.CLIENT" server connection channel and user "hannes"(you are prompted for the password):
/opt/mqm/samp/bin/amqscnxc -x '10.182.22.23(1415)' -c LDB1.CLIENT -u hannes LDB1
Sample AMQSCNXC start
Connecting to queue manager LDB1
using the server connection channel LDB1.CLIENT
on connection name 10.182.22.23(1415).
Enter password: ********
Connection established to queue manager LDB1
Sample AMQSCNXC end
If you can connect using this mechanism or "MQ Explorer" - then Splunk will connect too - as long as you use the same settings.
Overall all though it looks like your environment is working correctly. That means we can overcome any issue you may have.
Let me know how it goes! Thanks for your support.
I've done quite a bit the last few days. It running perfectly since Monday evening (after fixing the user and password issue). It's indexing 10+ million events from messages on MQ per day using a variery of response handlers. IBM Message Broker montiroring events(using the "BrokerEvents" response handler is at the top of that volume list.
Thanks for all the help! I'm quite certain the issue is now at our side - the amqscnxc
test gave some very useful hints especially .
I'm closing this issue, but as a sidenote: Do you know what the minimum of components of MQ client that are neccessary to install? Ref https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.1.0/com.ibm.mq.ins.doc/q008350_.htm
This is what I just used for my 9.0.0.4 installation (9.0.0.4-IBM-MQC-LinuxX64.tar.gz) on CentOS(ubuntu library names in V9.1 are the same)..
After accepting the license I installed the following:
MQSeriesRuntime-9.0.0-4.x86_64.rpm MQSeriesSDK-9.0.0-4.x86_64.rpm MQSeriesSamples-9.0.0-4.x86_64.rpm MQSeriesClient-9.0.0-4.x86_64.rpm MQSeriesJava-9.0.0-4.x86_64.rpm MQSeriesJRE-9.0.0-4.x86_64.rpm MQSeriesGSKit-9.0.0-4.x86_64.rpm MQSeriesMan-9.0.0-4.x86_64.rpm
eg.
rpm -ivh MQSeriesRuntime-9.0.0-4.x86_64.rpm MQSeriesSDK-9.0.0-4.x86_64.rpm MQSeriesSamples-9.0.0-4.x86_64.rpm MQSeriesClient-9.0.0-4.x86_64.rpm MQSeriesJava-9.0.0-4.x86_64.rpm MQSeriesJRE-9.0.0-4.x86_64.rpm MQSeriesGSKit-9.0.0-4.x86_64.rpm MQSeriesMan-9.0.0-4.x86_64.rpm
Not all of them are required.
If you do not want manual pages you can leave MQSeriesMan.
If you will not do any key management you can leave MQSeriesGSKit.
If you do not want the samples you can leave MQSeriesSamples.
You can also leave off the MQSeriesJava package if you have no intentions of using the IBM MQ series java classes.
But the rest you really should install for connectivity and building (MQSeriesRuntime-9.0.0-4.x86_64.rpm ,MQSeriesSDK-9.0.0-4.x86_64.rpm, MQSeriesClient-9.0.0-4.x86_64.rpm, MQSeriesJRE-9.0.0-4.x86_64.rpm)
If you are having issues connecting to your Queue Managers - post the problem here and I will assist where I can.
Hi,
Trying to get the add-on to work, but am getting the following error:
ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mq_ta/bin/mqchs.py" Stopping. Exception occurred in ChannelStatusPoller: Both user and password must be instances of str or bytes mqchs_stanza:mqchs://Test2_channel_status
Username is in form
_myuser
and pwd is in formmYs3Cr3tPwD42
Any pointers?