search

JPCERTCC / LogonTracer

Investigate malicious Windows logon by visualizing and analyzing Windows event log
Other
2.71k stars 442 forks source link

Data Fails to Load (EVTX via Web or Python) #24

Closed Broctets-and-Bytes closed 5 years ago

Broctets-and-Bytes commented 6 years ago

System appears up and running receiving 200 OK's on the server. Log looks to process with a couple of warnings at the end. Data doesn't load after uploading the log. Tried multiple times with multiple log sets, including the provided sample data. Also tried loading via local python in terminal and receive the same final error 'py:308: RuntimeWarning: Mean of empty slice.'. Page refreshes don't present data. Modified neo4j to have a longer timeout as well. Google Chrome shows nothing. Firefox shows 'Warning search failed!'.

Weblog file below:

[] Script start. 2018/07/17 11:24:33 [] Delete all nodes and relationships from this Neo4j database. [] Time zone is -5. [] Last record number is 23783. [] Start parsing the EVTX file. [] Parse the EVTX file Security.evtx. [] Now loading 100 records. [] Now loading 200 records. [] Now loading 300 records. [] Now loading 400 records. [] Now loading 500 records. [] Now loading 600 records. [] Now loading 700 records. [] Now loading 800 records. [] Now loading 900 records. [] Now loading 1000 records. [] Now loading 1100 records. [] Now loading 1200 records. [] Now loading 1300 records. [] Now loading 1400 records. [] Now loading 1500 records. [] Now loading 1600 records. [] Now loading 1700 records. [] Now loading 1800 records. [] Now loading 1900 records. [] Now loading 2000 records. [] Now loading 2100 records. [] Now loading 2200 records. [] Now loading 2300 records. [] Now loading 2400 records. [] Now loading 2500 records. [] Now loading 2600 records. [] Now loading 2700 records. [] Now loading 2800 records. [] Now loading 2900 records. [] Now loading 3000 records. [] Now loading 3100 records. [] Now loading 3200 records. [] Now loading 3300 records. [] Now loading 3400 records. [] Now loading 3500 records. [] Now loading 3600 records. [] Now loading 3700 records. [] Now loading 3800 records. [] Now loading 3900 records. [] Now loading 4000 records. [] Now loading 4100 records. [] Now loading 4200 records. [] Now loading 4300 records. [] Now loading 4400 records. [] Now loading 4500 records. [] Now loading 4600 records. [] Now loading 4700 records. [] Now loading 4800 records. [] Now loading 4900 records. [] Now loading 5000 records. [] Now loading 5100 records. [] Now loading 5200 records. [] Now loading 5300 records. [] Now loading 5400 records. [] Now loading 5500 records. [] Now loading 5600 records. [] Now loading 5700 records. [] Now loading 5800 records. [] Now loading 5900 records. [] Now loading 6000 records. [] Now loading 6100 records. [] Now loading 6200 records. [] Now loading 6300 records. [] Now loading 6400 records. [] Now loading 6500 records. [] Now loading 6600 records. [] Now loading 6700 records. [] Now loading 6800 records. [] Now loading 6900 records. [] Now loading 7000 records. [] Now loading 7100 records. [] Now loading 7200 records. [] Now loading 7300 records. [] Now loading 7400 records. [] Now loading 7500 records. [] Now loading 7600 records. [] Now loading 7700 records. [] Now loading 7800 records. [] Now loading 7900 records. [] Now loading 8000 records. [] Now loading 8100 records. [] Now loading 8200 records. [] Now loading 8300 records. [] Now loading 8400 records. [] Now loading 8500 records. [] Now loading 8600 records. [] Now loading 8700 records. [] Now loading 8800 records. [] Now loading 8900 records. [] Now loading 9000 records. [] Now loading 9100 records. [] Now loading 9200 records. [] Now loading 9300 records. [] Now loading 9400 records. [] Now loading 9500 records. [] Now loading 9600 records. [] Now loading 9700 records. [] Now loading 9800 records. [] Now loading 9900 records. [] Now loading 10000 records. [] Now loading 10100 records. [] Now loading 10200 records. [] Now loading 10300 records. [] Now loading 10400 records. [] Now loading 10500 records. [] Now loading 10600 records. [] Now loading 10700 records. [] Now loading 10800 records. [] Now loading 10900 records. [] Now loading 11000 records. [] Now loading 11100 records. [] Now loading 11200 records. [] Now loading 11300 records. [] Now loading 11400 records. [] Now loading 11500 records. [] Now loading 11600 records. [] Now loading 11700 records. [] Now loading 11800 records. [] Now loading 11900 records. [] Now loading 12000 records. [] Now loading 12100 records. [] Now loading 12200 records. [] Now loading 12300 records. [] Now loading 12400 records. [] Now loading 12500 records. [] Now loading 12600 records. [] Now loading 12700 records. [] Now loading 12800 records. [] Now loading 12900 records. [] Now loading 13000 records. [] Now loading 13100 records. [] Now loading 13200 records. [] Now loading 13300 records. [] Now loading 13400 records. [] Now loading 13500 records. [] Now loading 13600 records. [] Now loading 13700 records. [] Now loading 13800 records. [] Now loading 13900 records. [] Now loading 14000 records. [] Now loading 14100 records. [] Now loading 14200 records. [] Now loading 14300 records. [] Now loading 14400 records. [] Now loading 14500 records. [] Now loading 14600 records. [] Now loading 14700 records. [] Now loading 14800 records. [] Now loading 14900 records. [] Now loading 15000 records. [] Now loading 15100 records. [] Now loading 15200 records. [] Now loading 15300 records. [] Now loading 15400 records. [] Now loading 15500 records. [] Now loading 15600 records. [] Now loading 15700 records. [] Now loading 15800 records. [] Now loading 15900 records. [] Now loading 16000 records. [] Now loading 16100 records. [] Now loading 16200 records. [] Now loading 16300 records. [] Now loading 16400 records. [] Now loading 16500 records. [] Now loading 16600 records. [] Now loading 16700 records. [] Now loading 16800 records. [] Now loading 16900 records. [] Now loading 17000 records. [] Now loading 17100 records. [] Now loading 17200 records. [] Now loading 17300 records. [] Now loading 17400 records. [] Now loading 17500 records. [] Now loading 17600 records. [] Now loading 17700 records. [] Now loading 17800 records. [] Now loading 17900 records. [] Now loading 18000 records. [] Now loading 18100 records. [] Now loading 18200 records. [] Now loading 18300 records. [] Now loading 18400 records. [] Now loading 18500 records. [] Now loading 18600 records. [] Now loading 18700 records. [] Now loading 18800 records. [] Now loading 18900 records. [] Now loading 19000 records. [] Now loading 19100 records. [] Now loading 19200 records. [] Now loading 19300 records. [] Now loading 19400 records. [] Now loading 19500 records. [] Now loading 19600 records. [] Now loading 19700 records. [] Now loading 19800 records. [] Now loading 19900 records. [] Now loading 20000 records. [] Now loading 20100 records. [] Now loading 20200 records. [] Now loading 20300 records. [] Now loading 20400 records. [] Now loading 20500 records. [] Now loading 20600 records. [] Now loading 20700 records. [] Now loading 20800 records. [] Now loading 20900 records. [] Now loading 21000 records. [] Now loading 21100 records. [] Now loading 21200 records. [] Now loading 21300 records. [] Now loading 21400 records. [] Now loading 21500 records. [] Now loading 21600 records. [] Now loading 21700 records. [] Now loading 21800 records. [] Now loading 21900 records. [] Now loading 22000 records. [] Now loading 22100 records. [] Now loading 22200 records. [] Now loading 22300 records. [] Now loading 22400 records. [] Now loading 22500 records. [] Now loading 22600 records. [] Now loading 22700 records. [] Now loading 22800 records. [] Now loading 22900 records. [] Now loading 23000 records. [] Now loading 23100 records. [] Now loading 23200 records. [] Now loading 23300 records. [] Now loading 23400 records. [] Now loading 23500 records. [] Now loading 23600 records. [] Now loading 23700 records. [] Now loading 23800 records. [] Now loading 23900 records. [] Now loading 24000 records. [] Now loading 24100 records. [] Now loading 24200 records. [] Now loading 24300 records.logontracer.py:308: RuntimeWarning: Mean of empty slice. count_average = count_sum.mean(axis=0) /home/brock/.local/lib/python3.5/site-packages/numpy/core/_methods.py:73: RuntimeWarning: invalid value encountered in true_divide ret, rcount, out=ret, casting='unsafe', subok=False) [] Load finished. [] Total Event log is 24363. [] Calculate PageRank. [] Calculate ChangeFinder. [] Creating a graph data. [] Creation of a graph data finished. [] Script end. 2018/07/17 11:28:10

shu-tom commented 6 years ago

I can't reproduce the warning and this is not an error. The log may have been imported. Please check whether log is saved in the Neo4j at web GUI. http://[neo4j-server-ipaddress]:7474/

Broctets-and-Bytes commented 6 years ago

Thanks for the pointer. Didn't have anything other than debug log. Captured stdout to attached log. Looks like it may be stemming from an authentication related issue with bolt? Running on Ubuntu 16.04.4; Java 8; Neo Server 3.4.4 community.

neo4j.log

shu-tom commented 6 years ago

Neo4j's error log did not affect LogonTracer.

Can you see the imported data from the Neo4j web GUI as follows? image

Broctets-and-Bytes commented 6 years ago

I cannot, this is what I see in Neo4j: image while seeing this after refreshing LogonTracer and clicking any of the filtering options on the left: image

shu-tom commented 6 years ago

It's not reproducible on my environment. A reproducible test case would be very helpful.

NaykiSec commented 6 years ago

Hello @shu-tom , i think i'm facing the same problem.

I tried to import my security.evtx file and i have the same problem. Import seems to work fine except one error :

RuntimeWarning: Mean of empty slice. count_average = count_sum.mean(axis=0)

But after the import nothing is found on the Logontracer interface :8080 and same thing in the :7474 neo4js. (Same behavior than @Broctets-and-Bytes)

I then tried to import the sample Security.evtx that you are providing here and it's working great.

Maybe an interesting information : when i tried, the security.evtx has been imported from a French windows 10.

Get-Culture | Format-List -Property *
Parent                         : fr
LCID                           : 1036
KeyboardLayoutId               : 1036
Name                           : fr-FR
IetfLanguageTag                : fr-FR
DisplayName                    : Français (France)
NativeName                     : français (France)
EnglishName                    : French (France)
TwoLetterISOLanguageName       : fr
ThreeLetterISOLanguageName     : fra
ThreeLetterWindowsLanguageName : FRA
CompareInfo                    : CompareInfo - fr-FR
TextInfo                       : TextInfo - fr-FR
IsNeutralCulture               : False
CultureTypes                   : SpecificCultures, InstalledWin32Cultures, FrameworkCultures
NumberFormat                   : System.Globalization.NumberFormatInfo
DateTimeFormat                 : System.Globalization.DateTimeFormatInfo
Calendar                       : System.Globalization.GregorianCalendar
OptionalCalendars              : {System.Globalization.GregorianCalendar}
UseUserOverride                : True
IsReadOnly                     : False

I will try to use a Security.evtx from my lab where all the system is in english to see if it can be the root cause of our problem?

EDIT : Tested with Security.evtx coming for a Win 10 english : same problem usr/local/src/LogonTracer/logontracer.py:326: RuntimeWarning: Mean of empty slice. count_average = count_sum.mean(axis=0)

lanjianchun commented 6 years ago

do not use the 'localhost', you can check the html sources, F12 network console, there are some error!

aniketbhardwaj02 commented 6 years ago

Here is what I am getting, no errors that I can see but nothing is loaded in the http://:8080/#. I have reloaded as well but no change. I am using the docker instance.

[] Script start. 2018/09/26 19:35:10 [] Delete all nodes and relationships from this Neo4j database. [] Last record number is 35834. [] Start parsing the EVTX file. [] Parse the EVTX file Security.evtx. [] Now loading 100 records. [] Now loading 200 records. [] Now loading 300 records. [] Now loading 400 records. [] Now loading 500 records. [] Now loading 600 records. [] Now loading 700 records. [] Now loading 800 records. [] Now loading 900 records. [] Now loading 1000 records. [] Now loading 1100 records. [] Now loading 1200 records. [] Now loading 1300 records. [] Now loading 1400 records. [] Now loading 1500 records. [] Now loading 1600 records. [] Now loading 1700 records. [] Now loading 1800 records. [] Now loading 1900 records. [] Now loading 2000 records. [] Now loading 2100 records. [] Now loading 2200 records. [] Now loading 2300 records. [] Now loading 2400 records. [] Now loading 2500 records. [] Now loading 2600 records. [] Now loading 2700 records. [] Now loading 2800 records. [] Now loading 2900 records. [] Now loading 3000 records. [] Now loading 3100 records. [] Now loading 3200 records. [] Now loading 3300 records. [] Now loading 3400 records. [] Now loading 3500 records. [] Now loading 3600 records. [] Now loading 3700 records. [] Now loading 3800 records. [] Now loading 3900 records. [] Now loading 4000 records. [] Now loading 4100 records. [] Now loading 4200 records. [] Now loading 4300 records. [] Now loading 4400 records. [] Now loading 4500 records. [] Now loading 4600 records. [] Now loading 4700 records. [] Now loading 4800 records. [] Now loading 4900 records. [] Now loading 5000 records. [] Now loading 5100 records. [] Now loading 5200 records. [] Now loading 5300 records. [] Now loading 5400 records. [] Now loading 5500 records. [] Now loading 5600 records. [] Now loading 5700 records. [] Now loading 5800 records. [] Now loading 5900 records. [] Now loading 6000 records. [] Now loading 6100 records. [] Now loading 6200 records. [] Now loading 6300 records. [] Now loading 6400 records. [] Now loading 6500 records. [] Now loading 6600 records. [] Now loading 6700 records. [] Now loading 6800 records. [] Now loading 6900 records. [] Now loading 7000 records. [] Now loading 7100 records. [] Now loading 7200 records. [] Now loading 7300 records. [] Now loading 7400 records. [] Now loading 7500 records. [] Now loading 7600 records. [] Now loading 7700 records. [] Now loading 7800 records. [] Now loading 7900 records. [] Now loading 8000 records. [] Now loading 8100 records. [] Now loading 8200 records. [] Now loading 8300 records. [] Now loading 8400 records. [] Now loading 8500 records. [] Now loading 8600 records. [] Now loading 8700 records. [] Now loading 8800 records. [] Now loading 8900 records. [] Now loading 9000 records. [] Now loading 9100 records. [] Now loading 9200 records. [] Now loading 9300 records. [] Now loading 9400 records. [] Now loading 9500 records. [] Now loading 9600 records. [] Now loading 9700 records. [] Now loading 9800 records. [] Now loading 9900 records. [] Now loading 10000 records. [] Now loading 10100 records. [] Now loading 10200 records. [] Now loading 10300 records. [] Now loading 10400 records. [] Now loading 10500 records. [] Now loading 10600 records. [] Now loading 10700 records. [] Now loading 10800 records. [] Now loading 10900 records. [] Now loading 11000 records. [] Now loading 11100 records. [] Now loading 11200 records. [] Now loading 11300 records. [] Now loading 11400 records. [] Now loading 11500 records. [] Now loading 11600 records. [] Now loading 11700 records. [] Now loading 11800 records. [] Now loading 11900 records. [] Now loading 12000 records. [] Now loading 12100 records. [] Now loading 12200 records. [] Now loading 12300 records. [] Now loading 12400 records. [] Now loading 12500 records. [] Now loading 12600 records. [] Now loading 12700 records. [] Now loading 12800 records. [] Now loading 12900 records. [] Now loading 13000 records. [] Now loading 13100 records. [] Now loading 13200 records. [] Now loading 13300 records. [] Now loading 13400 records. [] Now loading 13500 records. [] Now loading 13600 records. [] Now loading 13700 records. [] Now loading 13800 records. [] Now loading 13900 records. [] Now loading 14000 records. [] Now loading 14100 records. [] Now loading 14200 records. [] Now loading 14300 records. [] Now loading 14400 records. [] Now loading 14500 records. [] Now loading 14600 records. [] Now loading 14700 records. [] Now loading 14800 records. [] Now loading 14900 records. [] Now loading 15000 records. [] Now loading 15100 records. [] Now loading 15200 records. [] Now loading 15300 records. [] Now loading 15400 records. [] Now loading 15500 records. [] Now loading 15600 records. [] Now loading 15700 records. [] Now loading 15800 records. [] Now loading 15900 records. [] Now loading 16000 records. [] Now loading 16100 records. [] Now loading 16200 records. [] Now loading 16300 records. [] Now loading 16400 records. [] Now loading 16500 records. [] Now loading 16600 records. [] Now loading 16700 records. [] Now loading 16800 records. [] Now loading 16900 records. [] Now loading 17000 records. [] Now loading 17100 records. [] Now loading 17200 records. [] Now loading 17300 records. [] Now loading 17400 records. [] Now loading 17500 records. [] Now loading 17600 records. [] Now loading 17700 records. [] Now loading 17800 records. [] Now loading 17900 records. [] Now loading 18000 records. [] Now loading 18100 records. [] Now loading 18200 records. [] Now loading 18300 records. [] Now loading 18400 records. [] Now loading 18500 records. [] Now loading 18600 records. [] Now loading 18700 records. [] Now loading 18800 records. [] Now loading 18900 records. [] Now loading 19000 records. [] Now loading 19100 records. [] Now loading 19200 records. [] Now loading 19300 records. [] Now loading 19400 records. [] Now loading 19500 records. [] Now loading 19600 records. [] Now loading 19700 records. [] Now loading 19800 records. [] Now loading 19900 records. [] Now loading 20000 records. [] Now loading 20100 records. [] Now loading 20200 records. [] Now loading 20300 records. [] Now loading 20400 records. [] Now loading 20500 records. [] Now loading 20600 records. [] Now loading 20700 records. [] Now loading 20800 records. [] Now loading 20900 records. [] Now loading 21000 records. [] Now loading 21100 records. [] Now loading 21200 records. [] Now loading 21300 records. [] Now loading 21400 records. [] Now loading 21500 records. [] Now loading 21600 records. [] Now loading 21700 records. [] Now loading 21800 records. [] Now loading 21900 records. [] Now loading 22000 records. [] Now loading 22100 records. [] Now loading 22200 records. [] Now loading 22300 records. [] Now loading 22400 records. [] Now loading 22500 records. [] Now loading 22600 records. [] Now loading 22700 records. [] Now loading 22800 records. [] Now loading 22900 records. [] Now loading 23000 records. [] Now loading 23100 records. [] Now loading 23200 records. [] Now loading 23300 records. [] Now loading 23400 records. [] Now loading 23500 records. [] Now loading 23600 records. [] Now loading 23700 records. [] Now loading 23800 records. [] Now loading 23900 records. [] Now loading 24000 records. [] Now loading 24100 records. [] Now loading 24200 records. [] Now loading 24300 records. [] Now loading 24400 records. [] Now loading 24500 records. [] Now loading 24600 records. [] Now loading 24700 records. [] Now loading 24800 records. [] Now loading 24900 records. [] Now loading 25000 records. [] Now loading 25100 records. [] Now loading 25200 records. [] Now loading 25300 records. [] Now loading 25400 records. [] Now loading 25500 records. [] Now loading 25600 records. [] Now loading 25700 records. [] Now loading 25800 records. [] Now loading 25900 records. [] Now loading 26000 records. [] Now loading 26100 records. [] Now loading 26200 records. [] Now loading 26300 records. [] Now loading 26400 records. [] Now loading 26500 records. [] Now loading 26600 records. [] Now loading 26700 records. [] Now loading 26800 records. [] Now loading 26900 records. [] Now loading 27000 records. [] Now loading 27100 records. [] Now loading 27200 records. [] Now loading 27300 records. [] Now loading 27400 records. [] Now loading 27500 records. [] Now loading 27600 records. [] Now loading 27700 records. [] Now loading 27800 records. [] Now loading 27900 records. [] Now loading 28000 records. [] Now loading 28100 records. [] Now loading 28200 records. [] Now loading 28300 records. [] Now loading 28400 records. [] Now loading 28500 records. [] Now loading 28600 records. [] Now loading 28700 records. [] Now loading 28800 records. [] Now loading 28900 records. [] Now loading 29000 records. [] Now loading 29100 records. [] Now loading 29200 records. [] Now loading 29300 records. [] Now loading 29400 records. [] Now loading 29500 records. [] Now loading 29600 records. [] Now loading 29700 records. [] Now loading 29800 records. [] Now loading 29900 records. [] Now loading 30000 records. [] Now loading 30100 records. [] Now loading 30200 records. [] Now loading 30300 records. [] Now loading 30400 records. [] Now loading 30500 records. [] Now loading 30600 records. [] Now loading 30700 records. [] Now loading 30800 records. [] Now loading 30900 records. [] Now loading 31000 records. [] Now loading 31100 records. [] Now loading 31200 records. [] Now loading 31300 records. [] Now loading 31400 records. [] Now loading 31500 records. [] Now loading 31600 records. [] Now loading 31700 records. [] Now loading 31800 records. [] Now loading 31900 records. [] Now loading 32000 records. [] Now loading 32100 records. [] Now loading 32200 records. [] Now loading 32300 records. [] Now loading 32400 records. [] Now loading 32500 records. [] Now loading 32600 records. [] Now loading 32700 records. [] Now loading 32800 records. [] Now loading 32900 records. [] Now loading 33000 records. [] Now loading 33100 records. [] Now loading 33200 records. [] Now loading 33300 records. [] Now loading 33400 records. [] Now loading 33500 records. [] Now loading 33600 records. [] Now loading 33700 records. [] Now loading 33800 records. [] Now loading 33900 records. [] Now loading 34000 records. [] Now loading 34100 records. [] Now loading 34200 records. [] Now loading 34300 records. [] Now loading 34400 records. [] Now loading 34500 records. [] Now loading 34600 records. [] Now loading 34700 records. [] Now loading 34800 records. [] Now loading 34900 records. [] Now loading 35000 records. [] Now loading 35100 records.

aniketbhardwaj02 commented 6 years ago

For above, after couple of hours, i was able to view results.

aniketbhardwaj02 commented 5 years ago

Do I need to download new docker version? If so, please let me know the steps.

shu-tom commented 5 years ago

Delete the downloaded docker image with the command "docker rmi", then download the new image with the command "docker pull jpcertcc/docker-logontracer".