pyparsing 3 compatibility

[michaelweiser]

When started with pyparsing 3, Peekaboo throws the following error:

2022-01-12 14:13:24,073 - root - (MainThread) - CRITICAL - Invalid expression, 'str' object has no attribu
te 'eval': {sample.type_declared}|filereport.mime_types <= {
'text/plain', 'inode/x-empty'} -> ignore

Additionally, when installed together with oletools>=0.60 our pip-generated wrapper refused to start us with the following dependency error:

    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (pyparsing 3.0.6 (de/lib/python3.8/site-pack
ages), Requirement.parse('pyparsing<3,>=2.1.0'), {'oletools'})

This is announced at install time as well:

ERROR: oletools 0.60 has requirement pyparsing<3,>=2.1.0, but you'll have pyparsing 3.0.6 which is incompatible.

[michaelweiser]

A workaround would be to add "pyparsing<3" similar to olevba.

[michaelweiser]

The first issue is gone with pyparsing 3.0.7, so seems to have been a regression that's been fixed.

Also, pip 21.3.1 correctly resolves the requirements of peekaboo and oletools into a pyparsing version < 3 when doing a fresh install:

$ pip uninstall oletools pyparsing && ~/pd/bin/pip install -e .
[...]
Collecting oletools>=0.54
  Using cached oletools-0.60-py2.py3-none-any.whl (968 kB)
[...]
Collecting pyparsing
  Using cached pyparsing-3.0.7-py3-none-any.whl (98 kB)
[...]
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
[...]
Installing collected packages: pyparsing, oletools, PeekabooAV
[...]
Successfully installed PeekabooAV-2.0 oletools-0.60 pyparsing-2.4.7
WARNING: You are using pip version 21.3.1; however, version 22.0.4 is available.

[michaelweiser]

The same happens in 18.04 vagrant:

vagrant@peekabooav:~$ /opt/peekaboo/bin/pip list |grep pip\\\|pyparsing\\\|oletools
oletools           0.60
pip                21.3.1
pyparsing          2.4.7
vagrant@peekabooav:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.6 LTS
Release:        18.04
Codename:       bionic

All this seems to be due to the 2020-resolver becoming the default in pip 20.3. Switching back to 20.2 makes the original message and version discrepancy appear again:

root@peekabooav:/vagrant/PeekabooAV# /opt/peekaboo/bin/pip install pip==20.2 && /opt/peekaboo/bin/pip uninstall oletools pyparsing && /opt/peekaboo/bin/pip install -e .
[...]
ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

oletools 0.60 requires pyparsing<3,>=2.1.0, but you'll have pyparsing 3.0.7 which is incompatible.
Successfully installed PeekabooAV oletools-0.60 pyparsing-3.0.7

Enabling the 2020-resolver as suggested with pip 20.2 further confirms this by making resolution work again:

root@peekabooav:/vagrant/PeekabooAV# /opt/peekaboo/bin/pip install pip==20.2 && /opt/peekaboo/bin/pip uninstall oletools pyparsing && /opt/peekaboo/bin/pip --use-feature=2020-resolver install -e .
[...]
Installing collected packages: pyparsing, oletools
Successfully installed oletools-0.60 pyparsing-2.4.7
WARNING: You are using pip version 20.2; however, version 21.3.1 is available.
You should consider upgrading via the '/opt/peekaboo/bin/python3 -m pip install --upgrade pip' command.

So the whole issue seems to be moot if pip is current enough which it automatically becomes when using our installer because virtualenv automatically upgrades pip. In other scenarios pip just needs to be upgraded (pip install -U pip).

So I'll close here for now. We can look at this again if problems are reported.