process_evtx and other python modules do not work on MS Windows 10, Autopsy 4.17.0, Python 3.9

commandline-be commented 3 years ago

Short on time i meant to run Autopsy and the Autopsy-Plugins on MS WIndows since these have installers.

binary installer cannot install into a custom folder, only installs in the folder for the admin user
Python Modules throw all kind of warnings and errors

Two moduels of intereset at this time (with Python 3.9)

Process_EVTX: null
Process_EVTX_By_EventID: null

Running the modules from the commandline shows for both modules (one example only)

Parse_Evtx_By_EventID.py", line 42, in import jarray ModuleNotFoundError: No module named 'jarray'

commandline-be commented 3 years ago

markmckinnon commented 3 years ago

You cannot run the Autopsy Python modules outside of Autopsy. They will only run utilizing Autopsy. If a module has an exe file in it then you can run that passing it any parameters that are needed, these programs are specific to run with Autopsy so they may have issues running outside of begin called by Autopsy. The plugin modules will only run with jython 2.7 which is what Autopsy uses. As for the ingest module startup error when you ran the two (2) modules did you provide any input in the ingest options panel for them? That might be why you are getting that issue.

commandline-be commented 3 years ago

You cannot run the Autopsy Python modules outside of Autopsy. They will only run utilizing Autopsy. If a module has an exe file in it then you can run that passing it any parameters that are needed, these programs are specific to run with Autopsy so they may have issues running outside of begin called by Autopsy. The plugin modules will only run with jython 2.7 which is what Autopsy uses. As for the ingest module startup error when you ran the two (2) modules did you provide any input in the ingest options panel for them? That might be why you are getting that issue.

I'll try again. To my understanding the source renamed the .evtx files to include a hostname so, "security.evtx" now reads "hostname security.evtx" I assume that is where it breaks

markmckinnon commented 3 years ago

If the event logs have been renamed to something else you can also specify them in the other field and that should handle them being renamed.

commandline-be commented 3 years ago

If the event logs have been renamed to something else you can also specify them in the other field and that should handle them being renamed.

Sadly, no. In despair i also clicked on 'extract file(s)' to no avail.

What i do is i select 'other' and write hostname_security.evtx in the box below, result is the same.

To make sure i also ran 'repair' to make sure Autopsy is not broken

commandline-be commented 3 years ago

2021-02-15 13:57:24.974 org.sleuthkit.autopsy.ingest.IngestManager startIngestJob SEVERE: Error starting ParseEvtx ingest module for job 0 Traceback (most recent call last): File "C:\Users\myforensicuser\AppData\Roaming\autopsy\python_modules\Process_EVTX\ParseEvtx.py", line 159, in startUp if self.local_settings.getSetting('All') == 'true': AttributeError: 'NoneType' object has no attribute 'getSetting'

org.python.core.Py.AttributeError(Py.java:205)
org.python.core.PyObject.noAttributeError(PyObject.java:1013)
org.python.core.PyObject.__getattr__(PyObject.java:1008)
ParseEvtx$py.startUp$14(C:\Users\myforensicuser\AppData\Roaming\autopsy\python_modules\Process_EVTX\ParseEvtx.py:178)
ParseEvtx$py.call_function(C:\Users\myforensicuser\AppData\Roaming\autopsy\python_modules\Process_EVTX\ParseEvtx.py)
org.python.core.PyTableCode.call(PyTableCode.java:167)
org.python.core.PyBaseCode.call(PyBaseCode.java:307)
org.python.core.PyBaseCode.call(PyBaseCode.java:198)
org.python.core.PyFunction.__call__(PyFunction.java:482)
org.python.core.PyMethod.instancemethod___call__(PyMethod.java:237)
org.python.core.PyMethod.__call__(PyMethod.java:228)
org.python.core.PyMethod.__call__(PyMethod.java:218)
org.python.core.PyMethod.__call__(PyMethod.java:213)
org.python.core.PyObject._jcallexc(PyObject.java:3626)
org.python.proxies.ParseEvtx$ParseEvtxDbIngestModule$683.startUp(Unknown Source)
org.sleuthkit.autopsy.ingest.DataSourceIngestPipeline$PipelineModule.startUp(DataSourceIngestPipeline.java:192)
org.sleuthkit.autopsy.ingest.DataSourceIngestPipeline.startUp(DataSourceIngestPipeline.java:81)
org.sleuthkit.autopsy.ingest.IngestJobPipeline.startUpIngestPipelines(IngestJobPipeline.java:551)
org.sleuthkit.autopsy.ingest.IngestJobPipeline.start(IngestJobPipeline.java:515)
org.sleuthkit.autopsy.ingest.IngestJob.start(IngestJob.java:213)
org.sleuthkit.autopsy.ingest.IngestManager.startIngestJob(IngestManager.java:435)
org.sleuthkit.autopsy.ingest.IngestManager$StartIngestJobTask.call(IngestManager.java:930)
org.sleuthkit.autopsy.ingest.IngestManager$StartIngestJobTask.call(IngestManager.java:893)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)

2021-02-15 13:57:24.99 org.sleuthkit.autopsy.ingest.IngestManager startIngestJob SEVERE: Error starting ParseEvtxByEventID ingest module for job 0 Traceback (most recent call last): File "C:\Users\myforensicuser\AppData\Roaming\autopsy\python_modules\Process_EVTX_By_EventID\Parse_Evtx_By_EventID.py", line 158, in startUp if self.local_settings.getSetting('All') == 'true': AttributeError: 'NoneType' object has no attribute 'getSetting'

org.python.core.Py.AttributeError(Py.java:205)
org.python.core.PyObject.noAttributeError(PyObject.java:1013)
org.python.core.PyObject.__getattr__(PyObject.java:1008)
Parse_Evtx_By_EventID$py.startUp$14(C:\Users\myforensicuser\AppData\Roaming\autopsy\python_modules\Process_EVTX_By_EventID\Parse_Evtx_By_EventID.py:175)
Parse_Evtx_By_EventID$py.call_function(C:\Users\myforensicuser\AppData\Roaming\autopsy\python_modules\Process_EVTX_By_EventID\Parse_Evtx_By_EventID.py)
org.python.core.PyTableCode.call(PyTableCode.java:167)
org.python.core.PyBaseCode.call(PyBaseCode.java:307)
org.python.core.PyBaseCode.call(PyBaseCode.java:198)
org.python.core.PyFunction.__call__(PyFunction.java:482)
org.python.core.PyMethod.instancemethod___call__(PyMethod.java:237)
org.python.core.PyMethod.__call__(PyMethod.java:228)
org.python.core.PyMethod.__call__(PyMethod.java:218)
org.python.core.PyMethod.__call__(PyMethod.java:213)
org.python.core.PyObject._jcallexc(PyObject.java:3626)
org.python.proxies.Parse_Evtx_By_EventID$ParseEvtxByEventIDIngestModule$686.startUp(Unknown Source)
org.sleuthkit.autopsy.ingest.DataSourceIngestPipeline$PipelineModule.startUp(DataSourceIngestPipeline.java:192)
org.sleuthkit.autopsy.ingest.DataSourceIngestPipeline.startUp(DataSourceIngestPipeline.java:81)
org.sleuthkit.autopsy.ingest.IngestJobPipeline.startUpIngestPipelines(IngestJobPipeline.java:551)
org.sleuthkit.autopsy.ingest.IngestJobPipeline.start(IngestJobPipeline.java:515)
org.sleuthkit.autopsy.ingest.IngestJob.start(IngestJob.java:213)
org.sleuthkit.autopsy.ingest.IngestManager.startIngestJob(IngestManager.java:435)
org.sleuthkit.autopsy.ingest.IngestManager$StartIngestJobTask.call(IngestManager.java:930)
org.sleuthkit.autopsy.ingest.IngestManager$StartIngestJobTask.call(IngestManager.java:893)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)

2021-02-15 13:57:24.994 org.sleuthkit.autopsy.ingest.IngestManager startIngestJob

markmckinnon commented 3 years ago

Try this and see if it works.

Go into the following directories:

C:\Users\\AppData\Roaming\autopsy\config\IngestModuleSettings\org.sleuthkit.autopsy.casemodule.AddImageWizardIngestConfigPanel and C:\Users\\AppData\Roaming\autopsy\config\IngestModuleSettings\org.sleuthkit.autopsy.ingest.RunIngestModulesDialog

and delete all the files that start with org.python.proxies. These are the settings for the modules and because they have been changed and using the new method is why you are getting this error.

commandline-be commented 3 years ago

org.sleuthkit.autopsy.casemodule.AddImageWizardIngestConfigPanel contained 2 such files org.sleuthkit.autopsy.ingest.RunIngestModulesDialog contained 0 such files

Improvement i noticed is the filenames in other are now preserved.

can i delete of all of such org.python.proxies files, i notice there are plenty under autopsy\config\IngestModuleSettings

commandline-be commented 3 years ago

2021-02-15 15:35:40.143 org.sleuthkit.autopsy.ingest.IngestManager startIngestJob SEVERE: Error starting ParseEvtx ingest module for job 0 Traceback (most recent call last): File "C:\Users\myforensicuser\AppData\Roaming\autopsy\python_modules\Process_EVTX\ParseEvtx.py", line 159, in startUp if self.local_settings.getSetting('All') == 'true': AttributeError: 'NoneType' object has no attribute 'getSetting'

org.python.core.Py.AttributeError(Py.java:205)
org.python.core.PyObject.noAttributeError(PyObject.java:1013)
org.python.core.PyObject.__getattr__(PyObject.java:1008)
ParseEvtx$py.startUp$14(C:\Users\myforensicuser\AppData\Roaming\autopsy\python_modules\Process_EVTX\ParseEvtx.py:178)
ParseEvtx$py.call_function(C:\Users\myforensicuser\AppData\Roaming\autopsy\python_modules\Process_EVTX\ParseEvtx.py)
org.python.core.PyTableCode.call(PyTableCode.java:167)
org.python.core.PyBaseCode.call(PyBaseCode.java:307)
org.python.core.PyBaseCode.call(PyBaseCode.java:198)
org.python.core.PyFunction.__call__(PyFunction.java:482)
org.python.core.PyMethod.instancemethod___call__(PyMethod.java:237)
org.python.core.PyMethod.__call__(PyMethod.java:228)
org.python.core.PyMethod.__call__(PyMethod.java:218)
org.python.core.PyMethod.__call__(PyMethod.java:213)
org.python.core.PyObject._jcallexc(PyObject.java:3626)
org.python.proxies.ParseEvtx$ParseEvtxDbIngestModule$559.startUp(Unknown Source)
org.sleuthkit.autopsy.ingest.DataSourceIngestPipeline$PipelineModule.startUp(DataSourceIngestPipeline.java:192)
org.sleuthkit.autopsy.ingest.DataSourceIngestPipeline.startUp(DataSourceIngestPipeline.java:81)
org.sleuthkit.autopsy.ingest.IngestJobPipeline.startUpIngestPipelines(IngestJobPipeline.java:551)
org.sleuthkit.autopsy.ingest.IngestJobPipeline.start(IngestJobPipeline.java:515)
org.sleuthkit.autopsy.ingest.IngestJob.start(IngestJob.java:213)
org.sleuthkit.autopsy.ingest.IngestManager.startIngestJob(IngestManager.java:435)
org.sleuthkit.autopsy.ingest.IngestManager$StartIngestJobTask.call(IngestManager.java:930)
org.sleuthkit.autopsy.ingest.IngestManager$StartIngestJobTask.call(IngestManager.java:893)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)

markmckinnon commented 3 years ago

Yes, you can. These files store the information from prior runs of the ingest modules so they will be populated with the information on your next run.

markmckinnon commented 3 years ago

When you installed the plugins how did you install them?

commandline-be commented 3 years ago

I used the installer, however, the installer does not permit me to specify the correct path so i copied them from one profile to the other appdata folder

markmckinnon commented 3 years ago

The installer should put it into the correct path for the user that is installing it. It assumes that if you are installing it then you will also have Autopsy installed and be the user. Which version of the installer?

commandline-be commented 3 years ago

Pretty sure i downloaded version 1.3 (155MB)

what i also tried was create a backup and copy a .zip download into the python_modules folder under APPDATA

commandline-be commented 3 years ago

The installer should put it into the correct path for the user that is installing it. It assumes that if you are installing it then you will also have Autopsy installed and be the user. Which version of the installer?

Thanks again. No specific version mentioned on the filename. 155MB size,

since i don't see mention of 4.17 under https://github.com/markmckinnon/autopsy i'd also want to make sure this should not cause any issue ?

markmckinnon commented 3 years ago

I just ran the evtx plugins on Autopsy 4.17 and they did not have issues.

commandline-be commented 3 years ago

I've reinstalled Autopsy before, reinstalled the plugins, same problem.

The logs show 'unknown source' and org.python.proxies.ParseEvtx$ParseEvtxDbIngestModule$559.startUp(Unknown Source) and AttributeError: 'NoneType' object has no attribute 'getSetting'

I don't see what i could be doing wrong. The evtx files are not easy to rename because they come from different machines.

My concern is when i reinstall Autopsy now i will loose all the configuration i put into it.

commandline-be commented 3 years ago

okay, ParseEvtx is running. I have no idea why. i did renamed 'hostname_security.evtx' to 'security.evtx'

What i did was

[ run ingest modules ] [ custom ... ] [ deselect all ] [ select process evtx ] [ select all log files ] [ select security.evtx ]

markmckinnon commented 3 years ago

If you want to use names that are not standard, ie: you have security.evtx from other machines and they are names host1_security.evtx then you could use the other check box and list all the event logs seperated by commas to look at those. If that does not work right deselect and select the "Other" checkbox.

commandline-be commented 3 years ago

Thus far i've only had success when i run ingest and select the 'custom' option. Then there is no error. Ah, i've not picked up the comma separation and listed them line by line.

I've started an ingest, no idea what the outcome will be.

commandline-be commented 3 years ago

Left it running all night, hung in the morning. The hourglass shows up as soon as the mousecursor enters the window. I notice it hangs easily after having executed process_evtx, while it was not fast before it did never hang.

Still cannot get my head around as to why it works when i define a custom profile yet not when i select predefined profiles.

shannaniggans commented 2 years ago

I am having what appears to be the same issue with the ParseEvtx module.

Windows 11 Pro / Autopsy 4.19.3 / ParseEvtx version 1.5 / Python 3.9.0
Autopsy tends to hang after the plugin has run, and the file view does not update the Data Artifacts.
Killing the Autopsy process and reopening the case shows the updated file view and artefacts under Data Artifacts.

2022-04-01 05:37:51.045 org.sleuthkit.autopsy.ingest.IngestJobPipeline logErrorMessage
SEVERE: ParseEvtx experienced an error during analysis (data source = LogicalFileSet1, objId = 9852, pipeline id = 0, ingest job id = 1)
java.lang.NullPointerException
    org.sleuthkit.datamodel.Blackboard.postArtifacts(Blackboard.java:95)
    org.sleuthkit.autopsy.ingest.IngestServices.fireModuleDataEvent(IngestServices.java:118)
    sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    java.lang.reflect.Method.invoke(Method.java:498)
    org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:190)
    org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:208)
    org.python.core.PyObject.__call__(PyObject.java:477)
    org.python.core.PyObject.__call__(PyObject.java:481)
    org.python.core.PyMethod.__call__(PyMethod.java:141)
    ParseEvtx$py.process$15(C:/Users/shanna/AppData/Roaming/autopsy/python_modules/Process_EVTX/ParseEvtx.py:482)
    ParseEvtx$py.call_function(C:/Users/shanna/AppData/Roaming/autopsy/python_modules/Process_EVTX/ParseEvtx.py)
    org.python.core.PyTableCode.call(PyTableCode.java:173)
    org.python.core.PyBaseCode.call(PyBaseCode.java:306)
    org.python.core.PyBaseCode.call(PyBaseCode.java:197)
    org.python.core.PyFunction.__call__(PyFunction.java:485)
    org.python.core.PyMethod.instancemethod___call__(PyMethod.java:237)
    org.python.core.PyMethod.__call__(PyMethod.java:228)
    org.python.core.PyMethod.__call__(PyMethod.java:218)
    org.python.core.PyMethod.__call__(PyMethod.java:213)
    org.python.core.PyObject._jcallexc(PyObject.java:3565)
    org.python.core.PyObject._jcall(PyObject.java:3598)
    org.python.proxies.ParseEvtx$ParseEvtxDbIngestModule$33.process(Unknown Source)
org.sleuthkit.autopsy.ingest.DataSourceIngestPipeline$DataSourcePipelineModule.executeTask(DataSourceIngestPipeline.java:93)    org.sleuthkit.autopsy.ingest.DataSourceIngestPipeline$DataSourcePipelineModule.executeTask(DataSourceIngestPipeline.java:72)
    org.sleuthkit.autopsy.ingest.IngestTaskPipeline.executeTask(IngestTaskPipeline.java:220)
    org.sleuthkit.autopsy.ingest.IngestJobPipeline.execute(IngestJobPipeline.java:1085)
    org.sleuthkit.autopsy.ingest.DataSourceIngestTask.execute(DataSourceIngestTask.java:41)
    org.sleuthkit.autopsy.ingest.IngestManager$ExecuteIngestJobTasksTask.run(IngestManager.java:1019)
    java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    java.util.concurrent.FutureTask.run(FutureTask.java:266)
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    java.lang.Thread.run(Thread.java:748)

shannaniggans commented 2 years ago

Should all be fixed in release 1.7 of the plugin - #50

markmckinnon / Autopsy-Plugins

process_evtx and other python modules do not work on MS Windows 10, Autopsy 4.17.0, Python 3.9 #33