lxml.etree._XPathEvaluatorBase._handle_result lxml.etree.XPathEvalError: Invalid predicate

[t3chn0m4g3]

Conpot is running now for a few minutes and I caught a exception in http server.py:

conpot_default         | 2018-08-16 18:05:10,572 New http session from 87.250.233.76 (fff71615-28fa-4842-8752-c175c2487789)
conpot_default         | 2018-08-16 18:05:10,572 HTTP/1.1 GET request from ('87.250.233.76', 38183): ('/robots.txt', [('Host', 'xxx.dynvpn.de'), ('Connection', 'Keep-Alive'), ('User-Agent', 'Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)'), ('From', 'support@search.yandex.ru'), ('Accept', '*/*')], None). fff71615-28fa-4842-8752-c175c2487789
conpot_default         | 2018-08-16 18:05:10,573 HTTP/1.1 response to ('87.250.233.76', 38183): 404. fff71615-28fa-4842-8752-c175c2487789
conpot_default         | ----------------------------------------
conpot_default         | Exception happened during processing of request from ('141.8.141.139', 41934)
conpot_default         | Traceback (most recent call last):
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 639, in process_request_thread
conpot_default         |     self.finish_request(request, client_address)
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 361, in finish_request
conpot_default         |     self.RequestHandlerClass(request, client_address, self)
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 696, in __init__
conpot_default         |     self.handle()
conpot_default         |   File "/usr/lib/python3.6/http/server.py", line 418, in handle
conpot_default         |     self.handle_one_request()
conpot_default         |   File "/usr/lib/python3.6/http/server.py", line 406, in handle_one_request
conpot_default         |     method()
conpot_default         |   File "/usr/lib/python3.6/site-packages/conpot/protocols/http/command_responder.py", line 803, in do_GET
conpot_default         |     '//http/htdocs/node[@name="' + self.path.partition('?')[0] + '"]'
conpot_default         |   File "src/lxml/etree.pyx", line 2278, in lxml.etree._ElementTree.xpath
conpot_default         |   File "src/lxml/xpath.pxi", line 359, in lxml.etree.XPathDocumentEvaluator.__call__
conpot_default         |   File "src/lxml/xpath.pxi", line 227, in lxml.etree._XPathEvaluatorBase._handle_result
conpot_default         | lxml.etree.XPathEvalError: Invalid predicate
conpot_default         | ----------------------------------------
conpot_default         | ----------------------------------------
conpot_default         | Exception happened during processing of request from ('5.255.251.12', 63266)
conpot_default         | Traceback (most recent call last):
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 639, in process_request_thread
conpot_default         |     self.finish_request(request, client_address)
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 361, in finish_request
conpot_default         |     self.RequestHandlerClass(request, client_address, self)
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 696, in __init__
conpot_default         |     self.handle()
conpot_default         |   File "/usr/lib/python3.6/http/server.py", line 418, in handle
conpot_default         |     self.handle_one_request()
conpot_default         |   File "/usr/lib/python3.6/http/server.py", line 406, in handle_one_request
conpot_default         |     method()
conpot_default         |   File "/usr/lib/python3.6/site-packages/conpot/protocols/http/command_responder.py", line 803, in do_GET
conpot_default         |     '//http/htdocs/node[@name="' + self.path.partition('?')[0] + '"]'
conpot_default         |   File "src/lxml/etree.pyx", line 2278, in lxml.etree._ElementTree.xpath
conpot_default         |   File "src/lxml/xpath.pxi", line 359, in lxml.etree.XPathDocumentEvaluator.__call__
conpot_default         |   File "src/lxml/xpath.pxi", line 227, in lxml.etree._XPathEvaluatorBase._handle_result
conpot_default         | lxml.etree.XPathEvalError: Invalid predicate
conpot_default         | ----------------------------------------
conpot_default         | ----------------------------------------
conpot_default         | Exception happened during processing of request from ('5.45.203.10', 61311)
conpot_default         | Traceback (most recent call last):
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 639, in process_request_thread
conpot_default         |     self.finish_request(request, client_address)
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 361, in finish_request
conpot_default         |     self.RequestHandlerClass(request, client_address, self)
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 696, in __init__
conpot_default         |     self.handle()
conpot_default         |   File "/usr/lib/python3.6/http/server.py", line 418, in handle
conpot_default         |     self.handle_one_request()
conpot_default         |   File "/usr/lib/python3.6/http/server.py", line 406, in handle_one_request
conpot_default         |     method()
conpot_default         |   File "/usr/lib/python3.6/site-packages/conpot/protocols/http/command_responder.py", line 803, in do_GET
conpot_default         |     '//http/htdocs/node[@name="' + self.path.partition('?')[0] + '"]'
conpot_default         |   File "src/lxml/etree.pyx", line 2278, in lxml.etree._ElementTree.xpath
conpot_default         |   File "src/lxml/xpath.pxi", line 359, in lxml.etree.XPathDocumentEvaluator.__call__
conpot_default         |   File "src/lxml/xpath.pxi", line 227, in lxml.etree._XPathEvaluatorBase._handle_result
conpot_default         | lxml.etree.XPathEvalError: Invalid predicate
conpot_default         | ----------------------------------------
conpot_default         | 2018-08-16 18:05:42,588 Session timed out: fff71615-28fa-4842-8752-c175c2487789
conpot_default         | 2018-08-16 18:07:15,579 New http session from 87.250.233.74 (b08a1ccb-2807-446d-8b99-b24a695b151f)
conpot_default         | 2018-08-16 18:07:15,579 HTTP/1.1 GET request from ('87.250.233.74', 60588): ('/log/sQuery/lib/cgi-bin/ssl.conf', [('Host', 'xxx.dynvpn.de'), ('Connection', 'Keep-Alive'), ('User-Agent', 'Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)'), ('From', 'support@search.yandex.ru'), ('Accept-Encoding', 'gzip,deflate'), ('Accept', '*/*')], None). b08a1ccb-2807-446d-8b99-b24a695b151f
conpot_default         | 2018-08-16 18:07:15,579 HTTP/1.1 response to ('87.250.233.74', 60588): 404. b08a1ccb-2807-446d-8b99-b24a695b151f
conpot_default         | 2018-08-16 18:07:45,593 Session timed out: b08a1ccb-2807-446d-8b99-b24a695b151f

[xandfury]

There is a syntax error somewhere here:

'//http/htdocs/node[@name="' + self.path.partition('?')[0] + '"]'

You just gotta love XML for it's versatility :wink: This should be an easy fix. Provided I know what exactly to throw at conpot.

[creolis]

Does this happen regularly? If yes, it would help us to see what URI is thrown at Conpot at the time of the exception. There's no helpful debug line there yet - so please could you add

logger.debug('Trying to handle GET to resource <%s>, initiated by %s', self.path, self.client_address)

at /protocols/http/command_responder.py:802, and start conpot in verbose mode (-v)? It should reveal the URI in question and show us how to handle the case, as I could not reproduce it throwing (semi-)random garbage at it.

Thanks :)

[t3chn0m4g3]

@creolis Thank you, I am on it.

[t3chn0m4g3]

Caught the exception again ...

conpot.log:

2018-08-29 08:11:37,194 Trying to handle GET to resource </axis-cgi/suse/cgi-bin/\"plog/principal.php?menue=>, initiated by ('87.250.244.4', 35140)
2018-08-29 08:11:38,284 Trying to handle GET to resource </axis-cgi/suse/cgi-bin/\"plog/principal.php?menue=>, initiated by ('141.8.141.139', 47130)
2018-08-29 08:11:39,379 Trying to handle GET to resource </axis-cgi/suse/cgi-bin/\"plog/principal.php?menue=>, initiated by ('77.88.9.135', 61979)

Exception on console:

conpot_default         | 2018-08-29 08:11:37,194 Trying to handle GET to resource </axis-cgi/suse/cgi-bin/\"plog/principal.php?menue=>, initiated by ('87.250.244.4', 35140)
conpot_default         | ----------------------------------------
conpot_default         | Exception happened during processing of request from ('87.250.244.4', 35140)
conpot_default         | Traceback (most recent call last):
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 651, in process_request_thread
conpot_default         |     self.finish_request(request, client_address)
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 361, in finish_request
conpot_default         |     self.RequestHandlerClass(request, client_address, self)
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 721, in __init__
conpot_default         |     self.handle()
conpot_default         |   File "/usr/lib/python3.6/http/server.py", line 418, in handle
conpot_default         |     self.handle_one_request()
conpot_default         |   File "/usr/lib/python3.6/http/server.py", line 406, in handle_one_request
conpot_default         |     method()
conpot_default         |   File "/usr/lib/python3.6/site-packages/conpot/protocols/http/command_responder.py", line 804, in do_GET
conpot_default         |     '//http/htdocs/node[@name="' + self.path.partition('?')[0] + '"]'
conpot_default         |   File "src/lxml/etree.pyx", line 2278, in lxml.etree._ElementTree.xpath
conpot_default         |   File "src/lxml/xpath.pxi", line 359, in lxml.etree.XPathDocumentEvaluator.__call__
conpot_default         |   File "src/lxml/xpath.pxi", line 227, in lxml.etree._XPathEvaluatorBase._handle_result
conpot_default         | lxml.etree.XPathEvalError: Invalid predicate
conpot_default         | ----------------------------------------
conpot_default         | 2018-08-29 08:11:38,284 Trying to handle GET to resource </axis-cgi/suse/cgi-bin/\"plog/principal.php?menue=>, initiated by ('141.8.141.139', 47130)
conpot_default         | ----------------------------------------
conpot_default         | Exception happened during processing of request from ('141.8.141.139', 47130)
conpot_default         | Traceback (most recent call last):
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 651, in process_request_thread
conpot_default         |     self.finish_request(request, client_address)
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 361, in finish_request
conpot_default         |     self.RequestHandlerClass(request, client_address, self)
conpot_default         |   File "/usr/lib/python3.6/socketserver.py", line 721, in __init__
conpot_default         |     self.handle()
conpot_default         |   File "/usr/lib/python3.6/http/server.py", line 418, in handle
conpot_default         |     self.handle_one_request()
conpot_default         |   File "/usr/lib/python3.6/http/server.py", line 406, in handle_one_request
conpot_default         |     method()
conpot_default         |   File "/usr/lib/python3.6/site-packages/conpot/protocols/http/command_responder.py", line 804, in do_GET
conpot_default         |     '//http/htdocs/node[@name="' + self.path.partition('?')[0] + '"]'
conpot_default         |   File "src/lxml/etree.pyx", line 2278, in lxml.etree._ElementTree.xpath
conpot_default         |   File "src/lxml/xpath.pxi", line 359, in lxml.etree.XPathDocumentEvaluator.__call__
conpot_default         |   File "src/lxml/xpath.pxi", line 227, in lxml.etree._XPathEvaluatorBase._handle_result
conpot_default         | lxml.etree.XPathEvalError: Invalid predicate
conpot_default         | ----------------------------------------
conpot_default         | 2018-08-29 08:11:39,379 Trying to handle GET to resource </axis-cgi/suse/cgi-bin/\"plog/principal.php?menue=>, initiated by ('77.88.9.135', 61979)

Hope that helps 😄

[creolis]

It really does .. sigh .. :) thanks!

[creolis]

Fixed by #402

[t3chn0m4g3]

Awesome! Thank you guys :)