Closed fixator10 closed 4 years ago
This may be related to #2821 , but I'd like to get a bit more info for looking into this. (especially since I thought we had a fallback strategy for this specific error)
If you have the following info:
I'll look into this more.
MongoDB shell version v4.2.0
MongoDB server version: 4.2.0
db.version(): 4.2.0
lsb_release -a
:
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.1 LTS
Release: 18.04
Codename: bionic
Same happened with 4.0.12 MongoDB
My apologies for not getting back to you on this sooner, all of the information below was more recently known than my response and I lost track of getting back to you on this.
Root cause: We require transaction or retry writes (the latter as a fallback) from the mongo instance in use. The failure didn't show with core prior to this, as prior to this, there would never be an attempt in core to set a value above the document level in the underling mongo structure.
As for handling this, we should document that requirement as for most users, this will mean use of mongos
Note: This does mean doing this should solve the issue in your specific case even prior to the proposed full remedies
There should also likely be detection of the lack of functionality in code prior to bot start-up to prevent mid execution breakage caused by lacking the functionality.
After converting DB to Replica set, exception traceback changed:
Traceback (most recent call last):
File "/home/fixator/Red-V3/lib/python3.7/site-packages/redbot/core/dev_commands.py", line 171, in _eval
result = await func()
File "<string>", line 3, in func
File "/home/fixator/Red-V3/lib/python3.7/site-packages/redbot/core/bot.py", line 529, in set_shared_api_tokens
group.update(tokens)
File "/home/fixator/Red-V3/lib/python3.7/site-packages/redbot/core/config.py", line 88, in __aexit__
await self.value_obj.set(self.raw_value)
File "/home/fixator/Red-V3/lib/python3.7/site-packages/redbot/core/config.py", line 534, in set
await super().set(value)
File "/home/fixator/Red-V3/lib/python3.7/site-packages/redbot/core/config.py", line 230, in set
await self.driver.set(self.identifier_data, value=value)
File "/home/fixator/Red-V3/lib/python3.7/site-packages/redbot/core/drivers/mongo.py", line 231, in set
session=session,
File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/fixator/Red-V3/lib/python3.7/site-packages/pymongo/collection.py", line 757, in insert_many
blk.ops = [doc for doc in gen()]
File "/home/fixator/Red-V3/lib/python3.7/site-packages/pymongo/collection.py", line 757, in <listcomp>
blk.ops = [doc for doc in gen()]
File "/home/fixator/Red-V3/lib/python3.7/site-packages/pymongo/collection.py", line 747, in gen
for document in documents:
File "/home/fixator/Red-V3/lib/python3.7/site-packages/redbot/core/drivers/mongo.py", line 304, in generate_documents_to_insert
document["_id"] = {"RED_uuid": uuid, "RED_primary_key": primary_keys + [pkey]}
TypeError: 'str' object does not support item assignment
This is still happens on either [p]set api
and bot.set_shared_api_tokens
I also have this error. After converting DB to Replica set
Experiencing same issue:
https://github.com/Cog-Creators/Red-DiscordBot/blob/8267ad9aabc15b9c339ae54a8a7acad9b2917508/redbot/core/drivers/mongo.py#L55 https://docs.mlab.com/faq/#why-am-i-getting-the-transaction-numbers-are-only-allowed-on-storage-engines-that-support-document-level-locking-error
I cannot use mlab as my mongodb server because of this error.
Traceback:
(venv) root@vultr:~/Red-kun# python -m redbot heroku --token $TOKEN --prefix !
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/root/venv/lib/python3.7/site-packages/redbot/__main__.py", line 179, in <module>
main()
File "/root/venv/lib/python3.7/site-packages/redbot/__main__.py", line 115, in main
loop.run_until_complete(red.maybe_update_config())
File "uvloop/loop.pyx", line 1451, in uvloop.loop.Loop.run_until_complete
File "/root/venv/lib/python3.7/site-packages/redbot/core/bot.py", line 147, in maybe_update_config
await self.db.schema_version.set(schema_version)
File "/root/venv/lib/python3.7/site-packages/redbot/core/config.py", line 165, in set
await self.driver.set(self.identifier_data, value=value)
File "/root/venv/lib/python3.7/site-packages/redbot/core/drivers/red_mongo.py", line 164, in set
upsert=True,
File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/root/venv/lib/python3.7/site-packages/pymongo/collection.py", line 1002, in update_one
session=session),
File "/root/venv/lib/python3.7/site-packages/pymongo/collection.py", line 858, in _update_retryable
_update, session)
File "/root/venv/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1280, in _retryable_write
return self._retry_with_session(retryable, func, s, None)
File "/root/venv/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1233, in _retry_with_session
return func(session, sock_info, retryable)
File "/root/venv/lib/python3.7/site-packages/pymongo/collection.py", line 854, in _update
retryable_write=retryable_write)
File "/root/venv/lib/python3.7/site-packages/pymongo/collection.py", line 824, in _update
retryable_write=retryable_write).copy()
File "/root/venv/lib/python3.7/site-packages/pymongo/pool.py", line 584, in command
user_fields=user_fields)
File "/root/venv/lib/python3.7/site-packages/pymongo/network.py", line 158, in command
parse_write_concern_error=parse_write_concern_error)
File "/root/venv/lib/python3.7/site-packages/pymongo/helpers.py", line 155, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: Transaction numbers are only allowed on storage engines that support document-level locking
connection string isn't the actual issue here, it's just masking the actual issue for people who don't have retry writes available. Actual issue is shown in Fixator's last comment here, and is related to sets in config which would require writes to multiple documents.
After more was looked into with this, and a lot of discussion about various ways of handling it, the answer is that we are going to need to drop mongo support. Probably in 3.2
There's more to it than just the API keys, this was just the part which pointed out the issue clearly. More details will be added in #3099 about this. Leaving this open as an informational issue for now.
For those wanting a DB backend for config, postgres is and and will continue to be supported
Other bugs
What were you trying to do?
Set token for service via
[p]set token
orbot.set_shared_api_tokens()
What were you expecting to happen?
Save token to database without exception
What actually happened?
PyMongo raises exception
pymongo.errors.OperationFailure
This also creates collection with name
%DBNAME%.Core.SHARED_API_TOKENS
(according to other configs, this should be justCore.SHARED_API_TOKENS
) Created collection is empty.How can we reproduce this issue?