Failed to run plugin Pcapreader. xpress

[zer0py2c]

Sometimes my cuckoo3 analysis failed, the debug log as follows:

2023-07-05 15:24:18 DEBUG [cuckoo.processing.worker]: Using event consumers. event_consumers=[<cuckoo.processing.post.eventconsumer.eventlogs.EventJSONFiles object at 0x7f99041f2b20>, <cuckoo.processing.post.eventconsumer.patternsigs.PatternFinder object at 0x7f99041f2b80>, <cuckoo.processing.post.eventconsumer.injection.ProcessInjection object at 0x7f99041f2c40>, <cuckoo.processing.post.eventconsumer.suspicious.SuspiciousEventScoring object at 0x7f99041f2ca0>] task_id=20230705-GMILGY_1 2023-07-05 15:24:18 DEBUG [cuckoo.processing.worker]: Chose translator for logfile. logfile=threemon.pb translator_class=<class 'cuckoo.processing.event.translate.threemon.reader.ThreemonReader'> task_id=20230705-GMILGY_1 2023-07-05 15:24:18 DEBUG [cuckoo.processing.worker]: Running processing plugin. plugin=Pcapreader stage=post task_id=20230705-GMILGY_1 2023-07-05 15:24:19 ERROR [cuckoo.processing.worker]: Failure during processing. error=Failed to run plugin Pcapreader. xpress task_id=20230705-GMILGY_1 Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/Cuckoo_processing-0.1.1-py3.8.egg/cuckoo/processing/worker.py", line 197, in _run_processing_instances data = instance.start() File "/usr/local/lib/python3.8/dist-packages/Cuckoo_processing-0.1.1-py3.8.egg/cuckoo/processing/post/network.py", line 312, in start for flow, ts, proto, sent, recv in r.process(): File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/reader.py", line 130, in process self.tcp and self.tcp.process(ts, ip, packet) File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/transport.py", line 143, in process s.process(ts, tcp, to_server) File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/transport.py", line 423, in process self.states[self.state](self, ts, tcp, to_server) File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/transport.py", line 342, in state_conn self.parent.handle( File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/transport.py", line 724, in handle while self.states[self.state](self, s, ts): File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/transport.py", line 678, in state_stream self.parent.handle( File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/protoparsers.py", line 285, in handle super(HttpsProtocol, self).handle(s, ts, protocol, sent, recv, tlsinfo) File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/protoparsers.py", line 262, in handle s, ts, protocols[protocol], req, self.parse_response(ts, recv), File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/protoparsers.py", line 198, in parse_response raise UnknownHttpEncoding(content_encoding) httpreplay.exceptions.UnknownHttpEncoding: xpress

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/Cuckoo_processing-0.1.1-py3.8.egg/cuckoo/processing/worker.py", line 231, in _handle_processing _run_processing_instances(processing_instances, ctx) File "/usr/local/lib/python3.8/dist-packages/Cuckoo_processing-0.1.1-py3.8.egg/cuckoo/processing/worker.py", line 205, in _run_processing_instances raise PluginError( File "/usr/local/lib/python3.8/dist-packages/Cuckoo_processing-0.1.1-py3.8.egg/cuckoo/processing/worker.py", line 197, in _run_processing_instances data = instance.start() File "/usr/local/lib/python3.8/dist-packages/Cuckoo_processing-0.1.1-py3.8.egg/cuckoo/processing/post/network.py", line 312, in start for flow, ts, proto, sent, recv in r.process(): File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/reader.py", line 130, in process self.tcp and self.tcp.process(ts, ip, packet) File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/transport.py", line 143, in process s.process(ts, tcp, to_server) File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/transport.py", line 423, in process self.states[self.state](self, ts, tcp, to_server) File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/transport.py", line 342, in state_conn self.parent.handle( File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/transport.py", line 724, in handle while self.states[self.state](self, s, ts): File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/transport.py", line 678, in state_stream self.parent.handle( File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/protoparsers.py", line 285, in handle super(HttpsProtocol, self).handle(s, ts, protocol, sent, recv, tlsinfo) File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/protoparsers.py", line 262, in handle s, ts, protocols[protocol], req, self.parse_response(ts, recv), File "/home/zer0py2c/.local/lib/python3.8/site-packages/httpreplay/protoparsers.py", line 198, in parse_response raise UnknownHttpEncoding(content_encoding) cuckoo.processing.errors.PluginError: Failed to run plugin Pcapreader. xpress 2023-07-05 15:24:19 ERROR [cuckoo.control]: Task post stage failed. task_id=20230705-GMILGY_1

[PhoenixSys]

Do you know how can i add ubunto vm to cuckoo ?

[Cryss76]

The exception is triggered by httpreplay while trying to process the pcap file here using the call r.process(), because httpreplay doesn't support Microsofts xpress compression Algorithm. (see here for the supported encodings/compressions) Therefore this is more a missing feature than a bug.

Depending on the view point this also isn't a problem in Cuckoo itself. As on the one site: An important part of the analysis failed and the analysis as a whole is not trustworthy. So its fine that it fails. On the other hand: The other parts of the analysis are worth while viewing and may bring some insight and therefore this is an error in Cuckoos exception-handling as a warning would suffice?

@zer0py2C did the analysis failed completely so that you can't view any part of the analysis? or are parts of it readable? @amadisson what do you think how the exception should be handle?

Also I don't have a clue how to catch and handle an exception in the "in part" of a for loop xD. Any pointers on how to do something like that?

[zer0py2c]

@Cryss76 Thanks for your replay! I also agree with your point of view. In fact, when httpreplay throws this exception, I can’t get the two files: report.json and tlsmaster.txt, and my task.json will record it as fatal_error state. But static analysis results, threemon logs, events/*.json, etc. all exist and are normal.

[zer0py2c]

@PhoenixSys I think this cuckoo document may help you: https://github.com/cuckoosandbox/cuckoo/blob/1b69675a6fe3524d154d1e6be1b79e014396ab22/docs/book/installation/guest/linux.rst I have tested successfully. 嘻嘻 :-)

[amadisson]

To support Microsoft xpress compression algorithm is missing feature in httpreplay. httpreplay shouldn't raise exceptions when raise_exceptions is False.

@Cryss76 Exception-handling as a warning would suffice. If network fails then other processing is still possible. I would handle exception in whole httpreplay part, not only for. If exception from outside then still continue working.

[GhostRock37]

hello, i have this problem too. how to solve this problem or disable this analysis with Pcapreader. xpress