search

log2timeline / plaso

Super timeline all the things
https://plaso.readthedocs.io
Apache License 2.0
1.71k stars 335 forks source link

log2timeline.py: when using artifact filters: Error binding parameter 1 - probably unsupported type #4675

Closed jleaniz closed 1 year ago

jleaniz commented 1 year ago

Describe the problem:

I encountered an error when processing files using --artifact_filters command-line option.

To Reproduce:

The version of Plaso you used: 20230311

The operating system you are running Plaso on (Not the operating system of the image/files you're trying to analyze): Ubuntu Jammy 22.04

Steps to reproduce the behavior including command line and arguments and output:

I get the following error when running log2timeline with the --artifact_filters option

 log2timeline.py --status_view none --hashers none --partitions all --volumes all --vss_stores none --artifact_filters AllUsersShellHistory,ApacheAccessLogs,BrowserCache,BrowserHistory,ChromeStorage,LinuxAuditLogs,LinuxAuthLogs,LinuxCronLogs,LinuxKernelLogFiles,LinuxLastlogFile,LinuxMessagesLogFiles,LinuxScheduleFiles,LinuxSysLogFiles,LinuxUtmpFiles,LinuxWtmp --parsers \!filestat --temporary_directory /tmp/79eb9d85f4bc42e495a283b52f458545/1685649207-62b59892b02c4f6ca29587ec37da6149-PlasoParserTask --logfile /mnt/turbinia/output/79eb9d85f4bc42e495a283b52f458545/1685649207-62b59892b02c4f6ca29587ec37da6149-PlasoParserTask/62b59892b02c4f6ca29587ec37da6149.log --unattended --storage_file /tmp/79eb9d85f4bc42e495a283b52f458545/1685649207-62b59892b02c4f6ca29587ec37da6149-PlasoParserTask/62b59892b02c4f6ca29587ec37da6149.plaso /tmp/79eb9d85f4bc42e495a283b52f458545/1685649207-62b59892b02c4f6ca29587ec37da6149-PlasoParserTask/uncompressed-1685649207
2023-06-01 20:13:55,880 [INFO] (MainProcess) PID:2122822 <data_location> Determined data location: /usr/share/plaso
2023-06-01 20:13:55,905 [INFO] (MainProcess) PID:2122822 <artifact_definitions> Determined artifact definitions path: /usr/share/artifacts
Checking availability and versions of dependencies.
[OPTIONAL]      unable to determine version information for: flor
[OK]

Source path             : /tmp/79eb9d85f4bc42e495a283b52f458545/1685649207-62b59892b02c4f6ca29587ec37da6149-PlasoParserTask/uncompressed-1685649207
Source type             : directory
Artifact filters        : AllUsersShellHistory, ApacheAccessLogs, BrowserCache, BrowserHistory, ChromeStorage, LinuxAuditLogs, LinuxAuthLogs, LinuxCronLogs, LinuxKernelLogFiles, LinuxLastlogFile, LinuxMessagesLogFiles, LinuxScheduleFiles, LinuxSysLogFiles, LinuxUtmpFiles, LinuxWtmp
Processing time         : 00:00:00

Processing started.
Unable to write to storage with error: Unable to query attribute container store with error: Error binding parameter 1 - probably unsupported type.

If i run the same command without the --artifact_filters option, processing completes without errors.

log2timeline.py --status_view none --hashers none --partitions all --volumes all --vss_stores none --parsers \!filestat --temporary_directory /tmp/79eb9d85f4bc42e495a283b52f458545/1685649207-62b59892b02c4f6ca29587ec37da6149-PlasoParserTask --logfile /mnt/turbinia/output/79eb9d85f4bc42e495a283b52f458545/1685649207-62b59892b02c4f6ca29587ec37da6149-PlasoParserTask/62b59892b02c4f6ca29587ec37da6149.log --unattended --storage_file /tmp/79eb9d85f4bc42e495a283b52f458545/1685649207-62b59892b02c4f6ca29587ec37da6149-PlasoParserTask/62b59892b02c4f6ca29587ec37da6149.plaso /tmp/79eb9d85f4bc42e495a283b52f458545/1685649207-62b59892b02c4f6ca29587ec37da6149-PlasoParserTask/uncompressed-1685649207
2023-06-01 20:16:54,566 [INFO] (MainProcess) PID:2123448 <data_location> Determined data location: /usr/share/plaso
2023-06-01 20:16:54,580 [INFO] (MainProcess) PID:2123448 <artifact_definitions> Determined artifact definitions path: /usr/share/artifacts
Checking availability and versions of dependencies.
[OPTIONAL]      unable to determine version information for: flor
[OK]

Source path             : /tmp/79eb9d85f4bc42e495a283b52f458545/1685649207-62b59892b02c4f6ca29587ec37da6149-PlasoParserTask/uncompressed-1685649207
Source type             : directory
Processing time         : 00:00:00

Processing started.
Processing completed.

Number of warnings generated while extracting events: 40.

Use pinfo to inspect warnings in more detail.

Expected behavior:

Processing should complete without errors when using artifact filters.

Debug output/tracebacks: log2timeline.py --version && log2timeline.py --troubles

plaso - log2timeline version 20230311 2023-06-01 20:13:12,388 [INFO] (MainProcess) PID:2122691 Determined data location: /usr/share/plaso Using Python version 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0]

Path: /usr/bin/log2timeline.py

plaso - log2timeline version 20230311

Checking availability and versions of dependencies. [OK] acstore version: 20230226 [OK] artifacts version: 20221219 [OK] bencode [OK] certifi version: 2022.12.07 [OK] cryptography version: 40.0.2 [OK] dateutil version: 2.8.2 [OK] defusedxml version: 0.7.1 [OK] dfdatetime version: 20230225 [OK] dfvfs version: 20221224 [OK] dfwinreg version: 20221218 [OK] dtfabric version: 20221218 [OPTIONAL] unable to determine version information for: flor [OK] future version: 0.18.2 [OK] lz4 version: 4.3.2 [OK] opensearchpy [OK] pefile version: 2023.2.7 [OK] psutil version: 5.9.4 [OK] pybde version: 20221031 [OK] pycreg version: 20221022 [OK] pyesedb version: 20220806 [OK] pyevt version: 20221022 [OK] pyevtx version: 20221101 [OK] pyewf version: 20140814 [OK] pyfsapfs version: 20221102 [OK] pyfsext version: 20220829 [OK] pyfsfat version: 20220925 [OK] pyfshfs version: 20220831 [OK] pyfsntfs version: 20221023 [OK] pyfsxfs version: 20220829 [OK] pyfvde version: 20220915 [OK] pyfwnt version: 20220922 [OK] pyfwsi version: 20230114 [OK] pylnk version: 20230205 [OK] pyluksde version: 20221103 [OK] pymodi version: 20221023 [OK] pymsiecf version: 20221024 [OK] pyolecf version: 20221024 [OK] pyparsing version: 2.4.7 [OK] pyphdi version: 20221025 [OK] pyqcow version: 20221124 [OK] pyregf version: 20221026 [OK] pyscca version: 20221027 [OK] pysigscan version: 20230109 [OK] pysmdev version: 20221028 [OK] pysmraw version: 20221028 [OK] pytsk3 version: 20221228 [OK] pytz [OK] pyvhdi version: 20221124 [OK] pyvmdk version: 20221124 [OK] pyvsgpt version: 20221029 [OK] pyvshadow version: 20221030 [OK] pyvslvm version: 20221025 [OK] redis version: 4.1.4 [OK] requests version: 2.30.0 [OK] six version: 1.16.0 [OK] urllib3 version: 1.26.15 [OK] xattr version: 0.10.1 [OK] xlsxwriter version: 3.0.8 [OK] yaml version: 5.4.1 [OK] yara version: 4.2.3 [OK] zmq version: 25.0.0

Additional context

artifacts_filter is used by Turbinia when running triage recipes.

joachimmetz commented 1 year ago

Getting a slightly different message Error binding parameter 2: type 'list' is not supported