Issue on creating incidents using LWP module

[bsivavani]

Hi all,

We have recently upgraded RHEL OS from 6.7 to 6.9. After upgrade we have observed that service-now incidents are not creating and failing with below error.

Can't connect to rstproxy.rwe.com:8080 (SSL connect attempt failed with unknown errorerror:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol)

We have perl-libwww-perl-5.833-5 version.

Could you please help on this error ?

[karenetheridge]

You should probably upgrade your libwww-perl and LWP-Protocol-https packages (corresponding to the perl modules LWP::UserAgent, and LWP::Protocol::https), for a start.

But it will be difficult to help much without being to trace the source of the errors. Have your programmers isolated it to a particular piece of code?

[bsivavani]

Thanks for the response.

I am able to connect to service-now url using attached script.

but I am seeing below error message while running incidents create script using attached code.

Can't connect to rwedev.service-now.com:443 (Connection timed out) LWP::Protocol::https::Socket: connect: Connection timed out at /usr/share/perl5/vendor_perl/LWP/Protocol/http.pm line 50. 500 Can't connect to rwedev.service-now.com:443 (Connection timed out) at test_inc.pl line 44.

I guess script is failing at code

invoke the SOAP call

my $result = $soap->call($method => @params);

Could you please help on where I am missing ?

[genio]

So we don't have the linked text, here's the first script:

#!/usr/bin/perl

use strict;
use warnings;
use SOAP::Lite 'trace', 'debug';
use Config::INI::Reader;
use Config::IniFiles;
use Cwd 'abs_path';
use File::Basename;
use File::Path;
use IO::Handle;
use LWP::UserAgent;
use Digest::MD5 'md5_hex';
use Data::Dumper;

my $cwd = abs_path($0);
$cwd = dirname($cwd);

# Load main config file
my $hrMainConf = Config::INI::Reader->read_file($cwd . '/config_lwp.ini') or die "Unable to open main config file: $!\n";
my $hrFieldMap;

        my $xmlContent;
        my $httpUa = new LWP::UserAgent;
        if ($hrMainConf->{'main'}->{'enable_proxy'}) {
                my $strProxyAuth = "";
                if (defined $hrMainConf->{'proxy'}->{'proxy_username'}) {
                        $strProxyAuth = "$hrMainConf->{'proxy'}->{'proxy_username'}:$hrMainConf->{'proxy'}->{'proxy_password'}\@";
                }
                $httpUa->proxy(['http','https'], "http://$strProxyAuth$hrMainConf->{'proxy'}->{'proxy_address'}:$hrMainConf->{'proxy'}->{'proxy_port'}");
        }

 $httpUa->protocols_allowed(['http','https']);
        my $strUrl = "https://$hrMainConf->{'servicenow'}->{'sn_username'}:$hrMainConf->{'servicenow'}->{'sn_password'}\@$hrMainConf->{'servicenow'}->{'sn_url'}/$ARGV{'page'}?WSDL";
        print "service-now url is $strUrl \n";
        #snt_log("Connecting to: " . $strUrl);
        my $httpReq = new HTTP::Request 'POST' => $strUrl;
        my $httpRet = $httpUa->request($httpReq) or die snt_log("Unable to read site: $!");

        if ($httpRet->is_success) {
                $xmlContent = $httpUa->request($httpReq)->content;
                                print "connection success $xmlContent \n";
        } else {
                #snt_log("Connection to Service-Now Failed: " . $httpRet->status_line);
                                print "connection failed.... $httpRet->status_line \n";
                exit;
        }

And the second:

#!/usr/bin/perl -w
use strict;
use warnings;
use SOAP::Lite 'trace', 'debug';
use Config::INI::Reader;
use Config::IniFiles;
use Cwd 'abs_path';
use File::Basename;
use File::Path;
use IO::Handle;
use LWP::UserAgent;
use Digest::MD5 'md5_hex';
use Data::Dumper;

$ENV{https_proxy} = "http://rproxy-neurathdc.rwe.com:8080"; # I assure you, it's SSL.
$ENV{HTTPS_PROXY} = "http://rproxy-neurathdc.rwe.com:8080"; # I assure you, it's SSL.

# specifying this subroutine, causes basic auth to use
# its credentials when challenged
sub SOAP::Transport::HTTP::Client::get_basic_credentials {
     # login as the itil user
     return 'srv_SOAPimporterNagios' => '...';
}
# declare the SOAP endpoint here

my $soap = SOAP::Lite
     -> proxy('https://rdev.service-now.com/incident.do?SOAP');

# calling the insert function
my $method = SOAP::Data->name('insert')
     ->attr({xmlns => 'http://www.service-now.com/'});

# create a new incident with the following short_description and category
my @params = ( SOAP::Data->name(short_description => 'test alert please ignore') );
   push(@params, SOAP::Data->name(category => 'Hardware') );

print "parameters are @params";

# invoke the SOAP call
my $result = $soap->call($method => @params);
print "result is $result";

[shadowcat-mst]

This sounds like a case of "you've upgraded a box to a version of openssl that's trying to use a version of SSL that your proxy doesn't like"

[genio]

Also, I know this is very hand-wavy, but SOAP is just... all kinds of no fun. You can get away from SOAP and use their REST API:

https://docs.servicenow.com/bundle/geneva-servicenow-platform/page/integrate/inbound_rest/task/t_GetStartedCreateInt.html

This would simplify your client code a great deal, no longer having to deal with SOAP::Lite.

[genio]

I'm going to close this issue out for now as it doesn't appear to be an issue with LWP. Note that it's listed in these other places:

https://stackoverflow.com/questions/50761495/ssl23-get-server-hellounknown-protocol-while-running-perl-script

https://support.nagios.com/forum/viewtopic.php?f=16&t=48406

Please let us know if you find evidence of it being a bug in LWP and we'll be happy to re-open this. For now, though, it does seem like a problem with SSL/TLS versions.

Thanks, Chase

[bsivavani]

@genio I can't use REST API at this moment. Its already in go-live.

Incident creation not working only after RHEL server upgrade to 6.9, before it was working fine.

current version details on server RHEL - 6.9 perl-libwww-perl-5.833-5 LWP::UserAgent - 6.34 LWP::Protocol::https - 6.07 IO::Socket::SSL - 2.056

[genio]

Take all things Perl out of the mix and test on the same box. Use the proxy with cURL:

curl --trace --proxy https://rproxy-neurathdc.rwe.com:8080 -L https://rdev.service-now.com/incident.do?SOAP

[bsivavani]

@genio proxy with https showing ssl error [root@XXXX tmp]# curl --trace --proxy https://rproxy-neurathdc.rwe.com:8080 -L https://rdev.service-now.com/incident.do?SOAP curl: (35) SSL connect error [root@XXXX tmp]#

Proxy with http showing the html code.

[genio]

Can you show us the output, please?

[bsivavani]

using https [root@XXXX tmp]# curl --trace --proxy https://rproxy-neurathdc.rwe.com:8080 -L https://r.service-now.com/incident.do?SOAP curl: (35) SSL connect error [root@XXXX tmp]#

[bsivavani]

using http proxy

[root@XXXX tmp]# curl --trace --proxy http://rproxy-neurathdc.rwe.com:8080 -L https://rdev.service-now.com/incident.do?SOAP
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
  <!-- FileName: index.html
         Language: [en]
    -->
  <!--Head-->
  <head>
    <meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
    <meta http-equiv="X-UA-Compatible" content="IE=7" />
    <title>RWEST Web Gateway - Notification</title>
    <script src="/mwg-internal/de5fs23hu73ds/files/javascript/sw.js" type="text/javascript" ></script>
    <!--
          <link rel="stylesheet" href="/mwg-internal/de5fs23hu73ds/files/default/stylesheet.css" />
        -->
    <style type="text/css">
      body, td, th, p { font: normal 14px/18px Tahoma, Arial, sans-serif; }
      td, th, p { font-size: 100%; }

      body {
        margin: 0;
        padding: 0;
        height: auto;
      }

      a {
        color: #418dbe;
        text-decoration: none;
      }

      #serviceNav,
      #footer,
      #lightview,
      #lightviewError {
        display: none;
      }

      a img {
        border: none;
      }

etc....

[genio]

Gah. That's my fault. curl -v instead of curl --trace. We want to see the request and response sequence, not the output.

[bsivavani]

Please find below.. proxyuser and proxypwd used to connect snow url

[root@XXXX tmp]# curl -v --proxy https://proxyuser:proxypwd@rproxy-neurathdc.rwe.com:8080 -L https://rdev.service-now.com/incident.do?SOAP
* About to connect() to proxy rproxy-neurathdc.rwe.com port 8080 (#0)
*   Trying 10.88.253.20... connected
* Connected to rproxy-neurathdc.rwe.com (10.88.253.20) port 8080 (#0)
* Establish HTTP proxy tunnel to rdev.service-now.com:443
* Proxy auth using Basic with user 'proxyuser'
> CONNECT rdev.service-now.com:443 HTTP/1.1
> Host: rdev.service-now.com:443
> Proxy-Authorization: Basic dWk0MTMyMjA6U3VtbWVyMjEx
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Proxy-Connection: Keep-Alive
>
< HTTP/1.0 200 Connection established
<
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=*.service-now.com,O=ServiceNow,L=San Diego,ST=California,C=US
*       start date: Jan 19 18:40:12 2018 GMT
*       expire date: Oct 01 19:10:11 2018 GMT
*       common name: *.service-now.com
*       issuer: CN=Entrust Certification Authority - L1K,OU="(c) 2012 Entrust, Inc. - for authorized use only",OU=See www.entrust.net/legal-terms,O="Entrust, Inc.",C=US
> GET /incident.do?SOAP HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: rdev.service-now.com
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Set-Cookie: JSESSIONID=D2D67F7FC8E8EECE51B8A72B1976ED25; Path=/; HttpOnly;Secure
* Authentication problem. Ignoring this.
< WWW-Authenticate: BASIC realm="Service-now"
< Content-Length: 0
< Date: Mon, 18 Jun 2018 18:53:49 GMT
< Server: ServiceNow
< Set-Cookie: BIGipServerpool_rwedev=3498103306.53566.0000; path=/; Httponly; Secure
< Strict-Transport-Security: max-age=63072000; includeSubDomains
< Connection: close
<
* Closing connection #0
[root@XXXX tmp]#

[dod38fr]

@bsivavani I'm sorry to bring bad news. You forgot to mask the Authentication value in your previous message. Now the user and password used in the curl query are compromised: anybody can read this value with base64 -d. You must change the password used in the curl command as soon as possible.

[bsivavani]

@dod38fr Don't worry.. the username and password used in curl command are not real authentication details and are dummy values.

[bsivavani]

I am able to resolve the issue by updating the LWP module to 6.34 and adding below lines in script $ENV{https_proxy} = "http://proxyhost:8080"; $ENV{HTTPS_PROXY} = "http://proxyhost:8080"; $ENV{PERL_LWP_ENV_PROXY} = 1;

Thanks for all your support.