keyPattern not existing in DuplicteKeyError, causing `KeyError`

[jbkoh]

Hi umongo team,

I'm experiencing a bug inside umongo, but unfortunately I wasn't able to regenerate it outside my codebase. I'm hoping to get some advice on it without a regenerator.

I have a document like this:

class MyDocument(Document):
  aaa = ReferenceField(OtherDoc)
  bbb = ReferenceField(OtherDoc)
  ccc = StringField()
  ddd = ReferenceField()

  class Meta
    indexes = [
      pytmongo.IndexMode(
        [
          ("aaa", pymongo.ASCENDING),
          ("bbb", pymongo.ASCENDING),
          ("ccc", pymongo.ASCENDING),
          ("ddd", pymongo.ASCENDING),
        ]
      )
    ]

However, when I get duplicated key error, instead of getting a proper error message, from https://github.com/Scille/umongo/blob/master/umongo/frameworks/motor_asyncio.py#L192, I get

    keys = sorted(exc.details['keyPattern'].keys())
KeyError: 'keyPattern'

And actual exc.details is this, which doesn't have keyPattern field:

{'index': 0, 'code': 11000, 'errmsg': 'E11000 duplicate key error collection: mymodule.my_document index: aaa_1_bbb_1_ccc_1_ddd_1_eee_1 dup key: { : ObjectId(\'some_value\'), : ObjectId(\'some_value\'), : "some_value", : "some_value", : "some_value" }'}

I'm using: python 3.8.10 umongo 3.0.0 motor: 2.5.0 pymongo: 3.12.0 mongodb: 3.6.20

Please kindly share your thoughts. Thanks in advance.

As a possible solution, I'd love to move the line (https://github.com/Scille/umongo/blob/master/umongo/frameworks/motor_asyncio.py#L192) into the try clause right next to it, but not sure if it's a desired behavior.

[lafrech]

Thanks.

The fix you propose would hide the error.

The logic here relies on the underlying exception to return a "keyPattern" with the keys involved in the duplication error.

My guess is this has changed in pymongo/motor or most probably in mongo itself.

Do you think you could try older pymongo/motor versions and mongo versions and see what happens.

In the exception you're quoting, It's not clear to me whether or not the keys appear in some string that would need to be parsed. It would be nice to get a better structure than to parse that string, but I'm afraid that's how it is.

If the underlying layers don't provide a safe way to determine the duplicated keys, perhaps we could drop the feature and don't return the keys at all.

[jbkoh]

Thanks for the response! I upgraded mongodb version from 3.6.20 to 4.2.16 and the issue is gone, though I recreated all the indexes as well. I think it was a transient issue. I'm closing this one. Thanks!

[lafrech]

I suppose it is MongoDB related: the message changes depending on the version. Still surprised because the code here dates back to MongoDB 3.

Anyway, if someone stumbles upon this again, their investigations are welcome. We could try to adapt the code.