TxMongo: command SON(...) failed with 'not authorized' on remains to execute command

[tonal]

After upgrade pymongo to 3.7.0 break TxMongo

2018-07-05 03:30:03,714 [scrapy.utils.signal] ERROR: Error caught on signal handler: <bound method MongodbSaver.spider_opened of <remains.mongo_ext.MongodbSaver object at 0x7fad95940dd8>>
Traceback (most recent call last):
  File "/opt/pyenv/versions/3.6.6/envs/remains/lib/python3.6/site-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/opt/pyenv/versions/3.6.6/envs/remains/lib/python3.6/site-packages/twisted/python/failure.py", line 422, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/remains_grab/remains/remains/mongo_ext.py", line 56, in spider_opened
    item_cnt=self.items_scraped, page_count=self.response_count))
  File "/opt/pyenv/versions/3.6.6/envs/remains/lib/python3.6/site-packages/twisted/internet/defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/opt/pyenv/versions/3.6.6/envs/remains/lib/python3.6/site-packages/txmongo/database.py", line 67, in on_ok
    _check_command_response(response, msg, allowable_errors)
  File "/opt/pyenv/versions/3.6.6/envs/remains/lib/python3.6/site-packages/pymongo/helpers.py", line 155, in _check_command_response
    raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: TxMongo: command SON([('insert', 'tasks'), ('documents', [{'_id': ObjectId('5b3d668b7f7d193791c668c4'), 'type': 'site', 'name': 'Акваполис', 'load_start': datetime.datetime(2018, 7, 5, 3, 30, 3, 658310), 'load_end': datetime.datetime(2018, 7, 5, 3, 30, 3, 684830), 'status': 'start', 'msg': None, 'item_cnt': 0, 'page_count': 0}]), ('ordered', True), ('writeConcern', {})]) on namespace remains.$cmd failed with 'not authorized on remains to execute command { insert: "tasks", ordered: true, writeConcern: {}, $db: "remains" }'

After downgrade pymongo to 3.6.1 - work Ok.

[tonal]

This is shown only when connecting to mongo with authorization

conf = dict(
  mongodb= dict(
    uri='mongodb://USER:PASSWORD@localhost:27017/remains',
    db='remains'))
conn = yield ConnectionPool(conf['mongodb']['uri'])
mdb = conn[conf['mongodb']['db']]

when

[IlyaSkriblovsky]

We are using pymongo's internal function to check command result: pymongo.helpers._check_command_response. Seems like they changed it's behavior in 3.7... It is perfectly fine on their side since this function is not public. We are probably need to copy&paste such pymongo's private functions instead of just importing them because this is not the first time it is breaking.

[psi29a]

grumble

[IlyaSkriblovsky]

Trying to fix this in #233

[IlyaSkriblovsky]

Fixed by #233