Closed Bloggzy closed 1 year ago
This is not a bug, Timesketch can only import Plaso files up to the version installed on Timesketch. If Plaso files are generated with. newer version, there can be dozen of things in the Plaso files that Timesketch does not know how to handle, which is why you see the error message.
So the solution is to update Plaso on the Timesketch, e.g. by updating Timesketch.
Hope that helps
Hey @jaegeral, thanks for the response.
As far as I knew, I was using the most up to date version of Timesketch i.e. "TIMESKETCH_VERSION=latest".
Is there something obvious that I'm missing?
My docker-compose.yml file says:
version: "3.7"
services:
timesketch-web:
container_name: timesketch-web
image: us-docker.pkg.dev/osdfir-registry/timesketch/timesketch:${TIMESKETCH_VERSION}
And the config.env file says:
# Timesketch version to run. Latest is build from the master branch and a release
# number is build from a release tag. Using latest means that you are running
# the bleeding edge version and we cannot guarantee that it will not be broken.
TIMESKETCH_VERSION=latest
Yeah this is something that comes up from time to time, see: https://github.com/google/timesketch/discussions/2589 https://github.com/google/timesketch/discussions/2859
I added a few infos / data points in here: https://github.com/google/timesketch/pull/2866 on how to get the versions.
When I execute the following on the timesketch-web container:
sudo docker exec timesketch-web log2timeline.py --troubles
I get returned:
plaso - log2timeline version 20230311
And when I query, the version from the log2timeline/plaso container:
sudo docker run --rm -v /home/user/data:/data log2timeline/plaso psort --version
I get this returned:
plaso - psort version 20230717
@jaegeral, please forgive my ignorance, but is there a way to call log2timeline/plaso from inside the Timesketch Docker container?
So that you know you’re using the same version as will be ingesting the plaso file later on?
You mean docker in docker? I think that is not possible. But yeah your output in https://github.com/google/timesketch/issues/2865#issuecomment-1672726684 confirms, the file you uploaded was created with a to new Plaso version.
So one thing you can do is check which version you have installed in your Timesketch system and have Plaso only create the plaso file you will upload later with that very Plaso version. So you could process your evidence with:
docker pull log2timeline/plaso:20230311
(see: https://hub.docker.com/r/log2timeline/plaso/tags)
I saw the docker exec call to log2timeline.py and wondered if it extended further.
sudo docker exec timesketch-web log2timeline.py --troubles
But not to worry, it was just wishful thinking.
I’ve been using the older docker tag to work around it, I’ll add it to our documentation in case anyone else runs into the same issue.
Thanks for your efforts, much appreciated.
And to be clear, I understand that it is a problem and that it needs some better integration / documentation to not run into those situations, but at the moment we do not have a short result for this :-/
It’s all good, I’ll sure you’ll work out a better long term solution, it’s an awesome project, a few minor things like this will always crop up.
Describe the bug It appears that timesketch_importer doesn't like the plaso files generated by the 20230717 version of log2timeline/plaso, but is will process plaso files generated by older versions of log2timeline/plaso e.g. 20230311.
Actions: Having generated a .plaso (20230717) file using, and attempting to import it into Timesketch, I get the following error:
"OSError: Format version: 20230327 is too new and not yet supported, minimum supported version: 20230107"
To Reproduce Steps to reproduce the behavior:
sudo docker run --rm -v /home/user/data:/data log2timeline/plaso log2timeline --timezone 'UTC' --parsers 'win7_slow,!filestat' --storage_file /data/host01.plaso /data/host01
timesketch_importer -u testuser -p password --host http://127.0.0.1 --timeline_name timeline-test-01 --sketch_name sketch-test-01 /home/user/data/host01.plaso
Checking file upload status: .[FAIL]
Original filename: host01
File on disk: /usr/share/timesketch/upload/70da71c984844d499f0c57ffc8d3bd45
File size: 1.15GB
Uploaded by: testuser
Provider: CLI importer tool
Context: /home/user/.local/bin/timesketch_importer -u testuser -p password --host http://127.0.0.1 --timeline_name timeline-test-01 --sketch_name sketch-test-01 /home/user/data/host01.plaso
Data label: plaso
Status: fail
Total File Events: 0
Error message: Traceback (most recent call last): File "/usr/bin/pinfo.py", line 85, in <module> if not Main(): File "/usr/bin/pinfo.py", line 62, in Main tool.PrintStorageInformation() File "/usr/lib/python3/dist-packages/plaso/cli/pinfo_tool.py", line 1673, in PrintStorageInformation storage_reader = self._GetStorageReader(self._storage_file_path) File "/usr/lib/python3/dist-packages/plaso/cli/pinfo_tool.py", line 587, in _GetStorageReader storage_factory.StorageFactory.CreateStorageReaderForFile(path)) File "/usr/lib/python3/dist-packages/plaso/storage/factory.py", line 49, in CreateStorageReaderForFile return sqlite_reader.SQLiteStorageFileReader(path) File "/usr/lib/python3/dist-packages/plaso/storage/sqlite/reader.py", line 20, in __init__ self._store.Open(path=path) File "/usr/lib/python3/dist-packages/acstore/sqlite_store.py", line 999, in Open self._ReadAndCheckStorageMetadata(check_readable_only=True) File "/usr/lib/python3/dist-packages/plaso/storage/sqlite/sqlite_file.py", line 231, in _ReadAndCheckStorageMetadata self._CheckStorageMetadata( File "/usr/lib/python3/dist-packages/plaso/storage/sqlite/sqlite_file.py", line 53, in _CheckStorageMetadata super(SQLiteStorageFile, self)._CheckStorageMetadata( File "/usr/lib/python3/dist-packages/acstore/sqlite_store.py", line 199, in _CheckStorageMetadata raise IOError(( OSError: Format version: 20230327 is too new and not yet supported, minimum supported version: 20230107.
Expected behavior Would be to import the events, this can still be achieved, but you have to use an older version of log2timeline/plaso e.g. log2timeline/plaso:20230311
Desktop (please complete the following information):