search

log2timeline / dftimewolf

A framework for orchestrating forensic collection, processing and data export
Apache License 2.0
296 stars 72 forks source link

Error while fetching server API version #934

Open PolymathMonkey opened 2 days ago

PolymathMonkey commented 2 days ago

Hello,

somehow I am not able to upload to timesketch via dftimewolf. I call the command like:

dftimewolf plaso_ts /home/dirk --timesketch_endpoint https://127.0.0.1:5000 --timesketch_user XXXX --timesketch_password somepassword

But it throws an exception like it cant find the api version:

[2024-11-22 06:46:35,440] [dftimewolf          ] SUCCESS  dfTimewolf tool initialized with UUID: 42804ccf-f9e3-4246-8636-2fd666b135e8
[2024-11-22 06:46:35,454] [dftimewolf          ] INFO     Loading recipe plaso_ts...                                                                           
[2024-11-22 06:46:35,918] [dftimewolf          ] INFO     Loaded recipe plaso_ts with 3 modules
[2024-11-22 06:46:35,918] [dftimewolf          ] INFO     Running preflights...                                                                                
[2024-11-22 06:46:35,918] [dftimewolf          ] INFO     Setting up modules...                                                                                
[2024-11-22 06:46:35,918] [dftimewolf.state    ] INFO     Setting up module: FilesystemCollector                       
[2024-11-22 06:46:35,918] [dftimewolf.state    ] INFO     Setting up module: LocalPlasoProcessor                       
[2024-11-22 06:46:35,919] [dftimewolf.state    ] INFO     Setting up module: TimesketchExporter                                            
[2024-11-22 06:46:35,921] [dftimewolf.state    ] CRITICAL An unknown error occurred in module LocalPlasoProcessor: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
[2024-11-22 06:46:36,216] [TimesketchExporter  ] INFO     New sketch created: 8                                                                                

[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR    dfTimewolf encountered one or more errors:                                                                                                                                                                                                        06:46:36 [39/1643]
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR    1: error from dftimewolf: An unknown error occurred in module LocalPlasoProcessor: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR    Traceback (most recent call last):       
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 791, in urlopen
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR        response = self._make_request(     
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR                   ^^^^^^^^^^^^^^^^^^^     
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 497, in _make_request
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR        conn.request(                                                                                    
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/urllib3/connection.py", line 395, in request                                                                                                                                                                 
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR        self.endheaders()
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR      File "/usr/lib64/python3.12/http/client.py", line 1331, in endheaders
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR        self._send_output(message_body, encode_chunked=encode_chunked)
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR      File "/usr/lib64/python3.12/http/client.py", line 1091, in _send_output
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR        self.send(msg)                                                                                   
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR      File "/usr/lib64/python3.12/http/client.py", line 1035, in send
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR        self.connect()                                                                                   
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/docker/transport/unixconn.py", line 26, in connect
[2024-11-22 06:46:36,269] [dftimewolf.state    ] ERROR        sock.connect(self.unix_socket)                                                        
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR    FileNotFoundError: [Errno 2] No such file or directory
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR                                                                                                         
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR    During handling of the above exception, another exception occurred:                                  
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR                                                                                                         
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR    Traceback (most recent call last):
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR      File "/usr/lib/python3.12/site-packages/requests/adapters.py", line 486, in send            
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR        resp = conn.urlopen(
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR               ^^^^^^^^^^^^^
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 845, in urlopen
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR        retries = retries.increment(         
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR                  ^^^^^^^^^^^^^^^^^^         
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/urllib3/util/retry.py", line 470, in increment
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR        raise reraise(type(error), error, _stacktrace) 
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/urllib3/util/util.py", line 38, in reraise                                                                                                                                                                   
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR        raise value.with_traceback(tb)
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 791, in urlopen                                           
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR        response = self._make_request(
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR                   ^^^^^^^^^^^^^^^^^^^     
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 497, in _make_request
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR        conn.request(                                                                                    
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/urllib3/connection.py", line 395, in request
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR        self.endheaders()                                                                                
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR      File "/usr/lib64/python3.12/http/client.py", line 1331, in endheaders
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR        self._send_output(message_body, encode_chunked=encode_chunked)
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR      File "/usr/lib64/python3.12/http/client.py", line 1091, in _send_output
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR        self.send(msg)
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR      File "/usr/lib64/python3.12/http/client.py", line 1035, in send
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR        self.connect()
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/docker/transport/unixconn.py", line 26, in connect
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR        sock.connect(self.unix_socket)
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR    urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR    
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR    
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR    During handling of the above exception, another exception occurred:
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR    
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR    Traceback (most recent call last):
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/docker/api/client.py", line 223, in _retrieve_server_version
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR        return self.version(api_version=False)["ApiVersion"]
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/docker/api/daemon.py", line 181, in version
[2024-11-22 06:46:36,270] [dftimewolf.state    ] ERROR        return self._result(self._get(url), json=True)
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR                            ^^^^^^^^^^^^^^
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/docker/utils/decorators.py", line 44, in inner
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR        return f(self, *args, **kwargs)
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR               ^^^^^^^^^^^^^^^^^^^^^^^^
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/docker/api/client.py", line 246, in _get
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR        return self.get(url, **self._set_request_timeout(kwargs))
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR      File "/usr/lib/python3.12/site-packages/requests/sessions.py", line 602, in get
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR        return self.request("GET", url, **kwargs)
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR      File "/usr/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR        resp = self.send(prep, **send_kwargs)
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR      File "/usr/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR        r = adapter.send(request, **kwargs)
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR      File "/usr/lib/python3.12/site-packages/requests/adapters.py", line 501, in send
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR        raise ConnectionError(err, request=request)
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR    requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR    
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR    The above exception was the direct cause of the following exception:
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR    
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR    Traceback (most recent call last):
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR      File "/home/dirk/dftimewolf/dftimewolf/lib/state.py", line 325, in _SetupModuleThread
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR        self._RunModuleSetUp(module, **new_args)
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR      File "/home/dirk/dftimewolf/dftimewolf/lib/state.py", line 355, in _RunModuleSetUp
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR        module.SetUp(**new_args)
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR      File "/home/dirk/dftimewolf/dftimewolf/lib/processors/localplaso.py", line 104, in SetUp
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR        if not self._CheckDockerImage():
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR               ^^^^^^^^^^^^^^^^^^^^^^^^
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR      File "/home/dirk/dftimewolf/dftimewolf/lib/processors/localplaso.py", line 49, in _CheckDockerImage
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR        client = docker.from_env()  # type: ignore
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR                 ^^^^^^^^^^^^^^^^^
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/docker/client.py", line 94, in from_env
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR        return cls(
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR               ^^^^
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/docker/client.py", line 45, in __init__
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR        self.api = APIClient(*args, **kwargs)
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/docker/api/client.py", line 207, in __init__
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR        self._version = self._retrieve_server_version()
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR      File "/home/dirk/.local/lib/python3.12/site-packages/docker/api/client.py", line 230, in _retrieve_server_version
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR        raise DockerException(
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR    docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
[2024-11-22 06:46:36,271] [dftimewolf.state    ] ERROR    
[2024-11-22 06:46:36,271] [dftimewolf.state    ] CRITICAL One or more unexpected errors occurred.
[2024-11-22 06:46:36,271] [dftimewolf.state    ] CRITICAL Please consider opening an issue: https://github.com/log2timeline/dftimewolf/issues/new
[2024-11-22 06:46:36,272] [dftimewolf          ] CRITICAL Critical error found. Aborting.

I tested the upload to timesketch with the timesketch importer:

python timesketch_importer.py --host http://localhost:5000 -u XXXX -p XXXX /home/dirk/test/data/ultimatum.plaso
[2024-11-21 13:25:45,234] timesketch_api.config_assistance/WARNING No timesketch section in the config                                                         
[2024-11-21 13:25:45,234] timesketch_importer.importer_frontend/INFO Saving Credentials.                                                                       
[2024-11-21 13:25:45,278] timesketch_importer.importer_frontend/INFO Creating a client.                                                                                                                                                                                                                                       
[2024-11-21 13:25:45,509] timesketch_importer.importer_frontend/INFO Client created.                                                                           
[2024-11-21 13:25:45,509] timesketch_importer.importer_frontend/INFO Saving TS config.                                                                                                                                                                                                                                        
[2024-11-21 13:25:45,648] timesketch_importer.importer_frontend/INFO New sketch created: [7] New Sketch From Importer CLI                                      
What is the timeline name [ultimatum]: test uploa                                                                                                              
[2024-11-21 13:25:51,772] timesketch_importer.importer_frontend/INFO Uploading file.                                                                                                                                                                                                                                          
[2024-11-21 13:25:51,774] timesketch_importer.importer_frontend/INFO About to upload file.                                                                     
[2024-11-21 13:25:52,078] timesketch_importer.importer_frontend/INFO File upload completed.                                                                                                                                                                                                                                   
Checking file upload status: .[DONE]                                                                                                                           
Timeline uploaded to Timeline Id: 1.                                                                                                                           
Status of the index is: SUCCESS

Is there some way to solve this or how can I further debug this issue? Thanks

ramo-j commented 2 days ago

That error is coming from docker, which is the preferred way of using plaso. Do you have docker set up correctly and running?

PolymathMonkey commented 2 days ago

Yes I got docker up and running using this doc: https://docs.docker.com/engine/install/fedora/ .. So far it not made problems, so it makes me question what doe set up "correctly" mean in that context. Just to get it right