Open mamazinho opened 4 months ago
roman-right
commented
4 months ago Hi! Thank you for the catch! Could you please provide a minimal executable example module that I can run and see the problem. It will help me to fix the error much faster.
martincpt
commented
3 months ago Hi, I came here after my code raised RevisionIdWasChanged error when I tried to create a document with the same value on a unique index field. I wonder why RevisionIdWasChanged is raised upon a DuplicateKeyError. Can any of you explain this?
How should I distinct that what should I handle then?
Cheers!
martincpt
commented
2 months ago @roman-right I don't want to open another issue but could you please explain why is this behaviour implemented? Can't we just let the original DuplicateKeyError to be raised on an actual duplicate key error?
coneillpj
commented
2 months ago @roman-right I don't want to open another issue but could you please explain why is this behaviour implemented? Can't we just let the original
DuplicateKeyErrorto be raised on an actual duplicate key error?
I have the same question. Not sure if this is the intended behavior
tdoan2010
commented
1 month ago Problem:
beanie raises RevisionIdWasChanged exception instead of DuplicateKeyError.save() was called. Since save() will try to update the document if it's already there, it will raise the RevisionIdWasChanged exception due to the unique constraint, as can be seen from the source code.Solution:
save(), use insert(), then you will get the DuplicateKeyError when the unique constraint is violated.martincpt
commented
1 month ago @tdoan2010 Interesting. However, what should we use then when we updating an entry?
tdoan2010
commented
1 month ago @martincpt in that case, just call .save(). It won't throw the error when we update a document and the unique constraint is not violated. It works as expected in my application.
RempelOliveira
commented
1 month ago @tdoan2010 .save() causes RevisionIdWasChanged exception instead of DuplicateKeyError as well, it's because, .save() uses .update() as well, but, there is another problem, I have a pre-lookup with custom conditions to check if the record exists and the update methods also does this, In this case I have two searches after the update and it is not good for performance.
tdoan2010
commented
1 month ago @RempelOliveira yes, but only if you call .save() to create a document.
In my case, I called .insert() for creation and .save() for update. By doing so, I didn't get the RevisionIdWasChanged exception.
RempelOliveira
commented
1 week ago @tdoan2010
For me the exception also occurs when updating. Please share your code so we can see why the exception wasn't thrown for you?!
Hi, this fix is broken a testcase that dont brake in 1.23.6. From this PR, an exception " except DuplicateKeyError:
is raised on my POST route in .save() method
As you see below:
Runnings test with 1.23.6
Runnings test with 1.24.0
Anything changed between the two tests, only beanie version, I read that problem can occur with race condition code, but is not the case, the exception is raised on inside the POST I save an "Machine" (my entity) instance.
On test, machine is created:
My API only gets the "machine" and updates a field:
Model is a common Document model (TimestampedDocument is just a class with updated_at and created_at fields):
Factory also is a common factory:
All stack error
============================= test session starts ============================== platform linux -- Python 3.11.5, pytest-7.4.4, pluggy-1.4.0 rootdir: /workspace/cz-coordinator-api configfile: pyproject.toml plugins: anyio-3.7.1, asyncio-0.21.1, Faker-22.6.0, cov-4.1.0, httpx-0.26.0, fastapi-deps-0.2.3, celery-0.0.0 asyncio: mode=Mode.STRICT collected 1 item tests/drivers/test_api.py F [100%] =================================== FAILURES =================================== ___________________ test_post_session_status_history_success ___________________ data = OrderedDict([('_id', '36'), ('revision_id', Binary(b"\xa0Q\xb1V?\xceJ\x84\x92\xd8\xf1\xd3\xfd'\xc3U", 4)), ('created_a..., ('external_id', 'kjdvqxmfejcvHlaFvhlG'), ('last_appeal_token_at', datetime.datetime(2024, 2, 22, 2, 6, 36, 767000))]) args = (), kwargs = {} @wraps(fn) def wrapper(data, *args, **kwargs): try: > return fn(data, *args, **kwargs) /home/matheus/.cache/pypoetry/virtualenvs/cz-coordinator-api-Lp1F0DE3-py3.11/lib/python3.11/site-packages/mongomock_motor/patches.py:56: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Collection(Database(mongomock.MongoClient('localhost', 27017), 'coordinator'), 'Machine') data = OrderedDict([('_id', '36'), ('revision_id', Binary(b"\xa0Q\xb1V?\xceJ\x84\x92\xd8\xf1\xd3\xfd'\xc3U", 4)), ('created_a..., ('external_id', 'kjdvqxmfejcvHlaFvhlG'), ('last_appeal_token_at', datetime.datetime(2024, 2, 22, 2, 6, 36, 767000))]) session = None, ordered = True def _insert(self, data, session=None, ordered=True): if session: raise_not_implemented('session', 'Mongomock does not handle sessions yet') if not isinstance(data, Mapping): results = [] write_errors = [] num_inserted = 0 for index, item in enumerate(data): try: results.append(self._insert(item)) except WriteError as error: write_errors.append({ 'index': index, 'code': error.code, 'errmsg': str(error), 'op': item, }) if ordered: break else: continue num_inserted += 1 if write_errors: raise BulkWriteError({ 'writeErrors': write_errors, 'nInserted': num_inserted, }) return results if not all(isinstance(k, str) for k in data): raise ValueError('Document keys must be strings') if BSON: # bson validation BSON.encode(data, check_keys=True) # Like pymongo, we should fill the _id in the inserted dict (odd behavior, # but we need to stick to it), so we must patch in-place the data dict if '_id' not in data: data['_id'] = ObjectId() object_id = data['_id'] if isinstance(object_id, dict): object_id = helpers.hashdict(object_id) if object_id in self._store: > raise DuplicateKeyError('E11000 Duplicate Key Error', 11000) E pymongo.errors.DuplicateKeyError: E11000 Duplicate Key Error /home/matheus/.cache/pypoetry/virtualenvs/cz-coordinator-api-Lp1F0DE3-py3.11/lib/python3.11/site-packages/mongomock/collection.py:516: DuplicateKeyError During handling of the above exception, another exception occurred: self = Machine(id='36', revision_id=None, created_at=datetime.datetime(2017, 3, 5, 9, 19, 7, 719000), updated_at=datetime.dat...46, 791000), external_id='kjdvqxmfejcvHlaFvhlG', last_appeal_token_at=datetime.datetime(2024, 2, 22, 2, 6, 36, 767415)) ignore_revision = False, session = None, bulk_writer = None, skip_actions = [] skip_sync = None args = ({'$set': {'_id': '36', 'revision_id': UUID('a051b156-3fce-4a84-92d8-f1d3fd27c355'), 'created_at': datetime.datetime(2...0), 'external_id': 'kjdvqxmfejcvHlaFvhlG', 'last_appeal_token_at': datetime.datetime(2024, 2, 22, 2, 6, 36, 767415)}},) pymongo_kwargs = {'upsert': True} arguments = [{'$set': {'_id': '36', 'revision_id': UUID('a051b156-3fce-4a84-92d8-f1d3fd27c355'), 'created_at': datetime.datetime(2...atetime(2024, 2, 22, 2, 6, 36, 767415)}},