Sqlite3 Error - Database is locked

[jpoling-dswrx]

Running on Fedora 24 x64.

python EventMonkey.py -n t1 -p /Testing/ -o /Testing/

Enumerating Event Files: 100% |########################################################################################################################################################################################| Traceback (most recent call last): File "EventMonkey.py", line 150, in Main() File "EventMonkey.py", line 145, in Main options File "/home/user/EventMonkey/libem/WindowsEventManager.py", line 159, in init self._InitDb() File "/home/user/EventMonkey/libem/WindowsEventManager.py", line 181, in _InitDb WINEVENT_COLUMN_ORDER File "/home/user/EventMonkey/libem/DbHandler.py", line 54, in CreateTableFromMapping cursor.execute(string) sqlite3.OperationalError: database is locked

Let me know what else you may need from me to assist troubleshooting.

[dlcowen]

It's saying the database is locked, the database is just a sqlite data file its trying to write out.

Can you confirm the user you are running eventmonkey as has write permissions to /Testing/ and no other process is locking the file via lsof?

The only other reason I could think of would be if you had two instances of event monkey running to the same path.

[dlcowen]

Or what matt mentioned, try adding in --threads 1 to see if its a thread locking issue.

[jpoling-dswrx]

I am reading from/writing to a mounted (via cifs) network share, for which my current account (root) has permissions to read/write. Only one instance running at a time as well. Not ruling out that this could be a network issue if somehow the connection cut out during and caused an error.

Will try to thread parameter suggestion and see whats we gets...

[dlcowen]

thanks, just checking as the error is a sqlite3 exception being raised when trying to just write data to the sqlite database

[jpoling-dswrx]

Yep, rather straight forward exception, just not certain what would even cause that to lock. Nothing useful in dmesg either.

[dlcowen]

i think this is a symptom of python finding locks in linux but not windows so it happily breaks rules that don't exist on our test platform. I'm sure Matt will test it when he gets back.

[jpoling-dswrx]

Ran with only one thread. Same error, unfortunately.

[devgc]

I can't reproduce in Windows. I will try setting up a Fedora x64 machine to test.

[devgc]

Still unable to reproduce on Fedora x64. Also, delete all the logs in the EventMonkey/logs directory. I dont think that is the issue but I know there is another issue similar when those logs hit their max size. I need to find a solution to that as well.

[tester@localhost EventMonkey]$ python EventMonkey.py -n test -p /mnt/TestData/Evtx/ -o Output/
Enumerating Event Files: 100% |###################################################################################|
Total Records = 85662
Processing Event Files: 100% |####################################################################################|
[tester@localhost EventMonkey]$

[jpoling-dswrx]

Wonder if it has something to do with writing to a CIFS/Samba network share. Will try re-running locally and see what I get.

[jpoling-dswrx]

Works locally. Has to have something to with reading from and writing to a mounted CIFS/Samba share.

[jpoling-dswrx]

So, seems to only have problems writing to a mounted share. I can read from the share and write locally, but it freezes up when writing to a share. It creates the files on the share (name.db and name.db-journal), but seems to lock up and eventually fail with the previous error message.