trailofbits / sienna-locomotive

A user-friendly fuzzing and crash triage tool for Windows
https://blog.trailofbits.com/user-friendly-fuzzing-with-sienna-locomotive
GNU Affero General Public License v3.0
131 stars 24 forks source link

gui, harness: run winchecksec in separate thread #318

Closed woodruffw closed 5 years ago

woodruffw commented 5 years ago

This is almost working, although I'm getting some exceptions from pysqlite about threads:

Exception during reset or similar
Traceback (most recent call last):
  File "C:\Users\IEUser\AppData\Local\Programs\Python\Python36-32\lib\site-packages\sqlalchemy\pool.py", line 712, in _finalize_fairy
    fairy._reset(pool)
  File "C:\Users\IEUser\AppData\Local\Programs\Python\Python36-32\lib\site-packages\sqlalchemy\pool.py", line 883, in _reset
    pool._dialect.do_rollback(self)
  File "C:\Users\IEUser\AppData\Local\Programs\Python\Python36-32\lib\site-packages\sqlalchemy\engine\default.py", line 459, in do_rollback
    dbapi_connection.rollback()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 3424 and this is thread id 6032
Exception closing connection <sqlite3.Connection object at 0x04587620>
Traceback (most recent call last):
  File "C:\Users\IEUser\AppData\Local\Programs\Python\Python36-32\lib\site-packages\sqlalchemy\pool.py", line 712, in _finalize_fairy
    fairy._reset(pool)
  File "C:\Users\IEUser\AppData\Local\Programs\Python\Python36-32\lib\site-packages\sqlalchemy\pool.py", line 883, in _reset
    pool._dialect.do_rollback(self)
  File "C:\Users\IEUser\AppData\Local\Programs\Python\Python36-32\lib\site-packages\sqlalchemy\engine\default.py", line 459, in do_rollback
    dbapi_connection.rollback()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 3424 and this is thread id 6032

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\IEUser\AppData\Local\Programs\Python\Python36-32\lib\site-packages\sqlalchemy\pool.py", line 317, in _close_connection
    self._dialect.do_close(connection)
  File "C:\Users\IEUser\AppData\Local\Programs\Python\Python36-32\lib\site-packages\sqlalchemy\engine\default.py", line 465, in do_close
    dbapi_connection.close()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 3424 and this is thread id 6032

I'm not that familiar with Qt's threading model or SQLAlchemy, but these exceptions don't make much sense to me: the session that we use in db.Checksec.byExecutable is created internally, within (what I think is) the same thread. CCing @ehennenfent for ideas here.

Closes #306.

ehennenfent commented 5 years ago

Is this the only issue? Every time I've seen this message so far, it's because something else was crashing, and the exception handler was causing threading issues.

woodruffw commented 5 years ago

Those are the only exceptions I see, although that "Exception during reset or similar" line is suspect (I can't find it anywhere in our source).

woodruffw commented 5 years ago

Tried using a contextmanager here, with no luck. "Exception during reset or similar" is apparently an SQLAlchemy NullPool thing, so I'll try fiddling with our pooling strategy.

woodruffw commented 5 years ago

Alright, I think I solved it: it looks the problem was not closing the session in the cached checksec case. I've also added an explicit session.close() to the fallthrough case. My best guess is that the underlying problem was that the session (or some part of it) gets embedded into each Checksec instance via Base, making SQLite unhappy when the former instances pass over threads.

woodruffw commented 5 years ago

New problem in the non-cached case:

sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: checksec.hash [SQL: 'INSERT INTO checksec (hash, aslr, authenticode, cfg, "dynamicBase", "forceIntegrity", gs, "highEntropyVA", isolation, nx, rfg, "safeSEH", seh, path) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters: ('87e4b7f186b9bf1f7d95c77c63793f7a3a1a887d4967b7519053b911eadb5ea3', 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 1, 'C:\\Users\\IEUser\\Downloads\\fuzzgoat-master\\x64\\Release\\Fuzzgoat.exe')] (Background on this error at: http://sqlalche.me/e/gkpj)