search

sensiblecodeio / scraperwiki-python

ScraperWiki Python library for scraping and saving data
https://scraperwiki.com
BSD 2-Clause "Simplified" License
160 stars 69 forks source link

Error on `atexit` event when using only pdftoxml util function #102

Closed nick13jaremek closed 7 years ago

nick13jaremek commented 7 years ago

Hi there,

I am using scraperwiki due to its pdftoxml function, which is quite handy. However, when running some custom tests that depend on this function, I end up greeted with the following stacktrace right after my tests pass:

Error in sys.exitfunc:
Traceback (most recent call last):
  File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/path/to/virtualenv/local/lib/python2.7/site-packages/scraperwiki/sql.py", line 126, in commit_transactions
    if _State._transaction is not None:
AttributeError: 'NoneType' object has no attribute '_transaction'

I am not using any of the SQL-related functions in my test files. How is it that the _State class is None in a file-context function (commit_transactions)?

pwaller commented 7 years ago

It is possible (but surprising) the interpreter teardown has already deleted the object by this point. I'm afraid this package is not receving much maintenance. Without trying to understand the problem in too much depth, if you change the condition to if _State is not None and _State._transaction is not None - does it fix the problem?

Is the problem reproducible 100% of the time? If the above suggested fix works, I will accept a pull request for that modification.

nick13jaremek commented 7 years ago

@pwaller A similar fix to to the one you're proposing came to my mind as a simple workaround, but I wanted to check beforehand if this behaviour was expected.

Everytime I tried running my passing tests, the stacktrace outlined in my previous message appears.

I changed the culprint line with the following conditional statement, and the stacktrace appears no more.

  if _State and _State._transaction is not None:
      # condition logic

Will gladly send a pull request with this fix.

pwaller commented 7 years ago

I had a quick look at the CPython interpreter source but can't figure out why _State is being set to None by the time the atexit handler is called. Is threading perhaps involved? In any case, I think your fix is probably sufficient.