log2timeline/psteal broken in current cast-based SIFT Workstation

[chrisbensch]

I installed a fresh Ubuntu 22.04.3 workstation, installed cast v0.14.0 and installed SIFT via the exact command from the cast GitHub page "cast install teamdfir/sift-saltstack". After the install finished I rebooted the workstation and logged in. The terminal popped up and I simply ran "log2timeline.py" and received the following error:

$ log2timeline.py 
Traceback (most recent call last):
  File "/usr/bin/log2timeline.py", line 11, in <module>
    from plaso.cli import log2timeline_tool
  File "/usr/lib/python3/dist-packages/plaso/cli/log2timeline_tool.py", line 14, in <module>
    from plaso.cli import extraction_tool
  File "/usr/lib/python3/dist-packages/plaso/cli/extraction_tool.py", line 20, in <module>
    from plaso import parsers  # pylint: disable=unused-import
  File "/usr/lib/python3/dist-packages/plaso/parsers/__init__.py", line 63, in <module>
    from plaso.parsers import text_plugins
  File "/usr/lib/python3/dist-packages/plaso/parsers/text_plugins/__init__.py", line 4, in <module>
    from plaso.parsers.text_plugins import android_logcat
  File "/usr/lib/python3/dist-packages/plaso/parsers/text_plugins/android_logcat.py", line 78, in <module>
    class AndroidLogcatTextPlugin(
  File "/usr/lib/python3/dist-packages/plaso/parsers/text_plugins/android_logcat.py", line 87, in AndroidLogcatTextPlugin
    _INTEGER = pyparsing.Word(pyparsing.nums).set_parse_action(
AttributeError: '_WordRegex' object has no attribute 'set_parse_action'. Did you mean: 'setParseAction'?

---

$ psteal.py 
Traceback (most recent call last):
  File "/usr/bin/psteal.py", line 21, in <module>
    from plaso.cli import psteal_tool
  File "/usr/lib/python3/dist-packages/plaso/cli/psteal_tool.py", line 11, in <module>
    from plaso.cli import extraction_tool
  File "/usr/lib/python3/dist-packages/plaso/cli/extraction_tool.py", line 20, in <module>
    from plaso import parsers  # pylint: disable=unused-import
  File "/usr/lib/python3/dist-packages/plaso/parsers/__init__.py", line 63, in <module>
    from plaso.parsers import text_plugins
  File "/usr/lib/python3/dist-packages/plaso/parsers/text_plugins/__init__.py", line 4, in <module>
    from plaso.parsers.text_plugins import android_logcat
  File "/usr/lib/python3/dist-packages/plaso/parsers/text_plugins/android_logcat.py", line 78, in <module>
    class AndroidLogcatTextPlugin(
  File "/usr/lib/python3/dist-packages/plaso/parsers/text_plugins/android_logcat.py", line 87, in AndroidLogcatTextPlugin
    _INTEGER = pyparsing.Word(pyparsing.nums).set_parse_action(
AttributeError: '_WordRegex' object has no attribute 'set_parse_action'. Did you mean: 'setParseAction'?

---

Edit: I've also installed via cast in my docker-sift personal container and also WSLv2 under Windows 11, all the same error messages.

[ekristen]

Thanks for the report. I'm not sure there's anything directly that we can do, but I'll try and take a look this week. log2timeline is installed via deb packages directly from the authoring team, so it's it's throwing an error it's likely a bug in plaso.

[digitalsleuth]

Hi @chrisbensch , can you run the following and provide the output: pip list | grep pyparsing

[joachimmetz]

Thanks for the report. I'm not sure there's anything directly that we can do, but I'll try and take a look this week. log2timeline is installed via deb packages directly from the authoring team, so it's it's throwing an error it's likely a bug in plaso.

@ekristen this is not a Plaso issue but a SIFT issue, you are using a older and incompatible version of pyparsing. Gather key data first before you draw conclusions. See https://pyparsing-docs.readthedocs.io/en/latest/pyparsing.html

setParseAction(*fns, **kwargs) → pyparsing.core.ParserElement
Deprecated - use set_parse_action

[ekristen]

@joachimmetz thank your feedback. We currently aren't controlling the version of pyparsing, so we can help the user look into that.

[joachimmetz]

I assume the user or SIFT is installing multiple versions and messing up the PYTHONPATH given that Plaso specifically requires 3.x or later

See:

• https://github.com/log2timeline/plaso/blob/main/config/dpkg/control
• https://pyparsing-docs.readthedocs.io/en/latest/whats_new_in_3_0_0.html

[ekristen]

I checked we don't appear to be, it could be another tool or this specific user.

[digitalsleuth]

I have an idea of what might be causing this (aside from the obvious pyparsing issue), and a way to avoid the issue. I've just started some testing and will let you know the results in the morning.

[joachimmetz]

@digitalsleuth the issue is very simple, there is an older version of pyparsing that has precedence on the system, ways to avoid this are (1) don't mix dpkg and pip (2) use more hermetic builds like Docker

[digitalsleuth]

@joachimmetz Those are both acceptable solutions, and I've already been working on those since I first identified that pyparsing was the issue.

What I mean is that I'm attempting to accurately identify the underlying cause (ie. how did it happen in the first place), so we can avoid other potential issues and package conflicts.

As soon as I can get to my computer to check the results of my testing, I'll keep you updated.

Cheers!

[digitalsleuth]

So, it looks like the issue is being caused by python-evtx, regardless of whether plaso is installed via pip or dpkg.

2024-02-17 19:49:54,601 [salt.loaded.int.module.cmdmod:907 ][DEBUG   ][13] stdout: Collecting python-evtx
  Downloading python_evtx-0.7.4-py3-none-any.whl (35 kB)
Collecting configparser==4.0.2
  Downloading configparser-4.0.2-py2.py3-none-any.whl (22 kB)
Requirement already satisfied: zipp==1.0.0 in /usr/lib/python3/dist-packages (from python-evtx) (1.0.0)
Requirement already satisfied: six in /usr/lib/python3/dist-packages (from python-evtx) (1.16.0)
Collecting pyparsing==2.4.7
  Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting more-itertools==5.0.0
  Downloading more_itertools-5.0.0-py3-none-any.whl (52 kB)
Collecting hexdump==3.3
  Downloading hexdump-3.3.zip (12 kB)
Building wheels for collected packages: hexdump
  Building wheel for hexdump (setup.py): started
  Building wheel for hexdump (setup.py): finished with status 'done'
  Created wheel for hexdump: filename=hexdump-3.3-py3-none-any.whl size=8896 sha256=0dc904bf33058e235d1214f1d0c97fc70c8757f49f4f6517b9f9369330bfa4ff
  Stored in directory: /root/.cache/pip/wheels/26/28/f7/f47d7ecd9ae44c4457e72c8bb617ef18ab332ee2b2a1047e87
Successfully built hexdump
Installing collected packages: pyparsing, more-itertools, hexdump, configparser, python-evtx
  Attempting uninstall: pyparsing
    Found existing installation: pyparsing 3.0.9
    Uninstalling pyparsing-3.0.9:
      Successfully uninstalled pyparsing-3.0.9
  Attempting uninstall: more-itertools
    Found existing installation: more-itertools 8.10.0
    Uninstalling more-itertools-8.10.0:
      Successfully uninstalled more-itertools-8.10.0
Successfully installed configparser-4.0.2 hexdump-3.3 more-itertools-5.0.0 pyparsing-2.4.7 python-evtx-0.7.4

There are a few ways to address this, and I've been testing them all:

1. Put python-evtx in its own virtualenv
   • This is simple enough to do, but will only fix the pyparsing issue with this tool at this moment, not if other tools pin their requirement for pyparsing
2. Put plaso in its own python3 virtualenv, installed using pip
   • Same as above, but since plaso is the tool most affected by the pyparsing issue, it will have a longer-term positive effect
   • If placed in a venv, it will need to be more regularly maintained to ensure it's up to date
3. Use the plaso docker
   • This will resolve the issue for both, and will require less maintenance for the SIFT VM in the long run, as it will always pull the latest docker on each install / update

Since we use the GIFT repo already anyways, I would normally prefer to install plaso by way of the GIFT repo, which means option 1 normally would be my first choice. However, since the docker is already built, this might be a long term better option.

I'm running all solutions through the testing phase right now to ensure they get packaged and installed correctly, and that the tools all work after installation without conflict.

I'll let you know how those tests go, and will have a PR shortly.

[joachimmetz]

So, it looks like the issue is being caused by python-evtx, regardless of whether plaso is installed via pip or dpkg.

Neither python-evtx or plaso are to blame here. This is caused by the fact you're mixing pip and dpkg, try running python-evtx in a virtual env instead as you indicate to isolate the 2 different package managers from one and other. You are basically shooting yourself in your own foot here.

Put plaso in its own python3 virtualenv, installed using pip

This is not a supported installation method. If you decide to go this route against advice, make sure to test things thoroughly.

Use the plaso docker

This is the recommended installation method. And should continue to work even if GIFT decides to move to 24.04 (unless Docker itself if not compatible with 24.04).

[digitalsleuth]

@joachimmetz Respectfully, while I understand your viewpoint, and what your end goal is, I want you to understand I'm not attempting to place blame. My first goal is to attempt to identify the source of the pyparsing version conflict, and I have identified that. While plaso (installed via dpkg, as per the instructions in the plaso manual) installs its required version, python-evtx (installed via pip) then removes the version which gets installed by plaso, and subsequently installs pyparsing==2.4.7, as should be evident via the log I posted earlier. Stating the facts does not equate to blame, and I am not saying that the tools are at fault, I am merely stating that there is a conflicting requirement between the two.

I did not, at any point during this testing phase, install plaso via pip and dpkg, and am not mixing pip and dpkg for either of the tools. I installed plaso via the guidance provided in the instruction manual and allowed everything else to install as it is designed to be installed. After this test, plaso did not work, and the reason was because, during the installation of python-evtx via pip (which occurs chronologically after the install of plaso), it removed the installed pyparsing module (3.0.9) and installed 2.4.7 (not supported for the current version of plaso). The only two applications currently experiencing a pyparsing conflict are these two.

My second goal is to present all possible courses of action for developing an appropriate way forward. As you can see, I have weighed the three options presented above, provided ways to resolve the issues, and recommended a way forward.

Once tests are completed and I have a successful state file generated for all three courses of action, I'll add a new comment here.

In closing, I hope you don't feel as if either you, or plaso, are being targeted or attacked as a result of this unfortunate issue. This was definitely not my intent, and if my statements made it seem that way, then I apologize. I endeavour to keep my communications with everyone polite and professional, and if anything I've said has caused this to not be the case, please accept my apologies.

Cheers!

Corey

[joachimmetz]

So, it looks like the issue is being caused by python-evtx, regardless of whether plaso is installed via pip or dpkg.

Your use of language says otherwise you are not stating a facts, you are attributing the cause of the issue to the installation to python-evtx and the same for @ekristen "so it's it's throwing an error it's likely a bug in plaso."

From https://en.wiktionary.org/wiki/blame

To assert or consider that someone is the cause of something negative; to place blame, to attribute responsibility (for something negative or for doing something negative).

stating a fact would be to say that the python-evtx installs pyparsing==2.4.7 and that get's precedence in the PYTHONPATH on your installation which then causes an incompatible version to be invoked by plaso.

I installed plaso via the guidance provided in the instruction manual and allowed everything else to install as it is designed to be installed.

No instruction manual can account for what you do on your system. This is your responsibility.

[ekristen]

@chrisbensch this should be resolved in the latest release v2024.02.24.

[NassemKa]

@ekristen Hi Following up on the previous issue with log2timeline I have installed latest SIFT Workstation VM (sift-22.04-20240201). and the problem persists.Any suggestions for further troubleshooting or potential solutions?

[bmmojo]

@ekristen Same with NassemKa. This problem unfortunately is still there with version v0.14.30 (2024-05-06)

[ekristen]

@NassemKa @bmmojo we are looking into it.