EVTX parsing using --add via python script failing

[rj-chap]

Hello there! I have a docker instance running that is working fine with one host at a time. I can push an EVTX to the docker image using the python script locally via:

python logontracer.py --delete -e ./Security.evtx -z 0 -u neo4j -p password -s 192.168.50.196

I was able to parse a few different EVTXs like this without issue (each time using --delete). However, when I try to add data to neo4j, I receive an exception. After pulling in one EVTX file using the above method, I tried to ingest 7 more EVTX files using the following loop:

for file in `ls ./cylr/redacted-*-Security.evtx`; do python logontracer.py --add -e $file -z 0 -u neo4j -p password -s 192.168.50.196; done

Each iteration simply runs the script against the identified Security.evtx files, e.g.:

python logontracer.py --add -e ./cylr/redacted-dc-Security.evtx -z 0 -u neo4j -p password -s 192.168.50.196

When this ran, all 7 attempts ended up with the same exception:

[+] Neo4j Kernel version: 4.2.6
[+] Load cashe files.
[+] Last record number is 2369.
[+] Start parsing the EVTX file.
[+] Parse the EVTX file ./cylr/redacted-Security.evtx.
[+] Now loading 2300 records.
[+] Load finished.
[+] Total Event log is 2369.
[+] Filtered Event log is 145858.
[+] Create cache files.
[+] Calculate ChangeFinder.
[+] Calculate Hidden Markov Model.
[+] Calculate PageRank.
[+] Creating a graph data.
logontracer.py:1285: DeprecationWarning: The transaction.commit() method is deprecated, use graph.commit(transaction) instead
  tx.commit()
Traceback (most recent call last):
  File "/Users/rizzle/.pyenv/versions/3.7.2/lib/python3.7/site-packages/py2neo/client/http.py", line 443, in from_json
    content = json_loads(data, object_hook=JSONHydrant.json_to_packstream)
  File "/Users/rizzle/.pyenv/versions/3.7.2/lib/python3.7/json/__init__.py", line 361, in loads
    return cls(**kw).decode(s)
  File "/Users/rizzle/.pyenv/versions/3.7.2/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Users/rizzle/.pyenv/versions/3.7.2/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "logontracer.py", line 1857, in <module>
    main()
  File "logontracer.py", line 1842, in main
    parse_evtx(args.evtx)
  File "logontracer.py", line 1285, in parse_evtx
    tx.commit()
  File "/Users/rizzle/.pyenv/versions/3.7.2/lib/python3.7/site-packages/py2neo/compat.py", line 190, in f_
    return f(*args, **kwargs)
  File "/Users/rizzle/.pyenv/versions/3.7.2/lib/python3.7/site-packages/py2neo/database.py", line 1022, in commit
    return self.graph.commit(self)
  File "/Users/rizzle/.pyenv/versions/3.7.2/lib/python3.7/site-packages/py2neo/database.py", line 367, in commit
    summary = self.service.connector.commit(tx.ref)
  File "/Users/rizzle/.pyenv/versions/3.7.2/lib/python3.7/site-packages/py2neo/client/__init__.py", line 1378, in commit
    bookmark = cx.commit(tx)
  File "/Users/rizzle/.pyenv/versions/3.7.2/lib/python3.7/site-packages/py2neo/client/http.py", line 216, in commit
    rs = HTTPResponse.from_json(r.status, r.data.decode("utf-8"))
  File "/Users/rizzle/.pyenv/versions/3.7.2/lib/python3.7/site-packages/py2neo/client/http.py", line 445, in from_json
    raise_from(ProtocolError("Cannot decode response content as JSON"), error)
  File "<string>", line 3, in raise_from
py2neo.errors.ProtocolError: Cannot decode response content as JSON

Each time the exception occurred, the script was in the [+] Creating a graph data. stage for the respective file being ingested.

I don't see this error noted anywhere. I'm reviewing the tool for potential inclusion in an upcoming course, but I'll need my students to be able to push evtx files via the CLI.

Thanks!!

[rj-chap]

I'm also noticing that the Filtered Event log count can be much higher than the Total Event log count, which doesn't seem to make sense. Should I open a ticket for that?

[+] Total Event log is 2369.
[+] Filtered Event log is 145858.

[rj-chap]

I'm running your primary docker container under macOS:

docker image

jpcertcc/docker-logontracer   latest    845817a5c504   4 months ago   1.29GB

docker details

Client:
 Cloud integration: 1.0.17
 Version:           20.10.8
 API version:       1.41
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:55:20 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.8
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.6
  Git commit:       75249d8
  Built:            Fri Jul 30 19:52:31 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.9
  GitCommit:        e25210fe30a0a703442421b0f60afac609f950a3
 runc:
  Version:          1.0.1
  GitCommit:        v1.0.1-0-g4144b63
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

LTHOSTNAME
192.168.50.196

PATH
/usr/local/bin:/var/lib/neo4j/bin:/usr/local/openjdk-11/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

JAVA_HOME
/usr/local/openjdk-11

LANG
C.UTF-8

JAVA_VERSION
11.0.11+9

NEO4J_SHA256
b26217ae08ae93d8ae8d83d3da2db082bdcc67b3005309f562edfeffff8dad8c

NEO4J_TARBALL
neo4j-community-4.2.6-unix.tar.gz

NEO4J_EDITION
community

NEO4J_HOME
/var/lib/neo4j

PYTHON_VERSION
3.7.8

PYTHON_PIP_VERSION
20.1.1

PYTHON_GET_PIP_URL
https://github.com/pypa/get-pip/raw/eff16c878c7fd6b688b9b4c4267695cf1a0bf01b/get-pip.py

PYTHON_GET_PIP_SHA256
b3153ec0cf7b7bbf9556932aa37e4981c35dc2a2c501d70d91d2795aa532be79

Mounts
/LOGS
/var/lib/docker/volumes/983bec8c9bd84350cdb4d5379451b2fd6c5d2281fea7d6548d2b3f63daec1d72/_data

/DATA
/var/lib/docker/volumes/7bc3d72dd4053c22b80620285e5fdf8b0c7c4a41ff4847ac389d2146c960ec41/_data

Ports
7473/tcp
Not bound
7474/tcp
0.0.0.0:7474

7687/tcp
0.0.0.0:7687

8080/tcp
0.0.0.0:8080

ProductName:    Mac OS X
ProductVersion: 10.15.7
BuildVersion:   19H1323

I can provide a docker inspect or whatever else you'd like, but I cannot provide the actual EVTX files for obvious reasons.

[rj-chap]

As a note, I'm using Python 3.7.2 via pyenv. I can switch to a more preferred python version if one exists. Thanks!

[shu-tom]

I have not been able to reproduce this issue. What version of py2neo are you using?

[rj-chap]

I have not been able to reproduce this issue. What version of py2neo are you using?

Outside the docker image where, where I'm running the script, I'm running 2021.2.0:

py2neo             2021.2.0

[shu-tom]

Thank you for version info. But I still haven't been able to reproduce this issue. I've fixed the py2neo DeprecationWarning message, so I'm not sure if it's related to this issue, could you try the fixed version?