authenication failure with mongodb auth database

[ctow123]

One line description of the issue

fails with pymongo==3.10 ... authenication of mongodb is not working properly

Python script

pymongo.errors.OperationFailure: command listCollections requires authentication when using a django app with djongo and a mongodb that requires auth with said parameters, the following trace happens. see the code below where I seem to have found the problem and offer a solution

``` 'default': { 'ENGINE': 'djongo', 'NAME': 'applicationdb', 'USERNAME' : 'xxx', 'PASSWORD' : 'xxx', 'HOST': '127.0.0.1', 'PORT': 27017, 'AUTH_SOURCE': 'admin', 'AUTH_MECHANISM': 'SCRAM-SHA-256', from base.py why are the valid_settings only 2 fields there should be more like USER, PASSWORD, HOST, AUTHDB, etc. and these should be passed into connection_params so they can be used in get new connection cont. valid_settings = { 'NAME': 'name', 'ENFORCE_SCHEMA': 'enforce_schema', } connection_params = { 'name': 'djongo_test', 'enforce_schema': False } for setting_name, kwarg in valid_settings.items(): cont. def get_new_connection(self, connection_params): cont. self.client_connection = Database.connect(db=name, **connection_params) ``` #### Traceback File "manage.py", line 15, in execute_from_command_line(sys.argv) File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line utility.execute() File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv self.execute(*args, **cmd_options) File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 364, in execute output = self.handle(*args, **options) File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped res = handle_func(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 87, in handle executor = MigrationExecutor(connection, self.migration_progress_callback) File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 18, in __init__ self.loader = MigrationLoader(self.connection) File "/usr/local/lib/python3.7/site-packages/django/db/migrations/loader.py", line 49, in __init__ self.build_graph() File "/usr/local/lib/python3.7/site-packages/django/db/migrations/loader.py", line 212, in build_graph self.applied_migrations = recorder.applied_migrations() File "/usr/local/lib/python3.7/site-packages/django/db/migrations/recorder.py", line 73, in applied_migrations if self.has_table(): File "/usr/local/lib/python3.7/site-packages/django/db/migrations/recorder.py", line 56, in has_table return self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()) File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/introspection.py", line 48, in table_names return get_names(cursor) File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/introspection.py", line 43, in get_names return sorted(ti.name for ti in self.get_table_list(cursor) File "/usr/local/lib/python3.7/site-packages/djongo/introspection.py", line 47, in get_table_list for c in cursor.db_conn.list_collection_names() File "/usr/local/lib/python3.7/site-packages/pymongo/database.py", line 856, in list_collection_names for result in self.list_collections(session=session, **kwargs)] File "/usr/local/lib/python3.7/site-packages/pymongo/database.py", line 819, in list_collections _cmd, read_pref, session) File "/usr/local/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1464, in _retryable_read return func(session, server, sock_info, slave_ok) File "/usr/local/lib/python3.7/site-packages/pymongo/database.py", line 816, in _cmd **kwargs) File "/usr/local/lib/python3.7/site-packages/pymongo/database.py", line 766, in _list_collections session=tmp_session)["cursor"] File "/usr/local/lib/python3.7/site-packages/pymongo/database.py", line 630, in _command client=self.__client) File "/usr/local/lib/python3.7/site-packages/pymongo/pool.py", line 613, in command user_fields=user_fields) File "/usr/local/lib/python3.7/site-packages/pymongo/network.py", line 167, in command parse_write_concern_error=parse_write_concern_error) File "/usr/local/lib/python3.7/site-packages/pymongo/helpers.py", line 159, in _check_command_response raise OperationFailure(msg % errmsg, code, response) pymongo.errors.OperationFailure: command listCollections requires authentication

[Naumansh]

i am facing same problem with below settings:

DATABASES = {
    'default': {
        'ENGINE':'djongo',
        'NAME': 'admin',
        'USERNAME': 'KKK',
        'PASSWORD': 'KKK1',
        'HOST': '127.0.0.1',
        'PORT': 27017,
        'AUTH_SOURCE': 'admin',
    }
}

Django==2.2 django-jquery==3.1.0 djangorestframework==3.11.0 djongo==1.3.1 pymongo==3.10.1

[Naumansh]

@ctow123

I see you are setting up things to access multiple databases using dongo. I wanna do same thing but not very much sure how can i keep my params of the DATABASES map as variable and update them according to information received from response. I would really appreciate if you can guide me on how you are trying to do it. Thanks.

[ctow123]

@Naumansh if you downgrade djongo to 1.2.30 it fixed my problem and might fix yours. the problem is in the base.py and database.py files of djongo 1.3.1. the valid settings and connection params are not reading in database settings besides the name of your database as you can see in my original post on the issue above and get_new_connection therefore isn't passing the params to pymongo hence the error in the trace.

downgrade djongo to 1.2.30 and use these settings 'ENGINE':'djongo', 'NAME': 'admin', 'USERNAME': 'KKK', 'PASSWORD': 'KKK1', 'HOST': 'mongodb://username:password@127.0.0.1:27017', 'PORT': 27017, 'AUTH_SOURCE': 'admin', 'AUTH_MECHANISM': 'SCRAM-SHA-256', (if u have an older verision this may be SHA-1) ^obviously fill in username and password with your information

as for the multiple databases now quite sure what you mean. i only use one database but have multiple different collections (tables) in that database. I'm not sure why you would want another database. connecting to 2 databases and sending requests to each would require a lot more of a djongo rewrite and signifigant hours. i would recommend just using mongodb collections

[Naumansh]

@ctow123 Downgraded to djongo 1.2.3 with pymongo 3.10.1 Executing : python manage.py makemigrations Below are the updated config

DATABASES = { 'default': { 'ENGINE': 'djongo', 'ENFORCE_SCHEMA': False, 'NAME': 'accounts', 'HOST': 'mongodb://myuser:mypass@127.0.0.1:27017', 'PORT': 27017, 'USER': 'myuser', 'PASSWORD': 'mypass', 'AUTH_SOURCE': 'accounts', 'AUTH_MECHANISM': 'SCRAM-SHA-1' } }

Unfortunately still seeing same stack trace:

File "/Users/nauman.shahid/Nauman-data/Dev_Home/python/Venv/cyborg_lms_mvt/lib/python3.7/site-packages/pymongo/database.py", line 881, in collection_names nameOnly=True, kws)] File "/Users/nauman.shahid/Nauman-data/Dev_Home/python/Venv/cyborg_lms_mvt/lib/python3.7/site-packages/pymongo/database.py", line 819, in list_collections _cmd, read_pref, session) File "/Users/nauman.shahid/Nauman-data/Dev_Home/python/Venv/cyborg_lms_mvt/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1464, in _retryable_read return func(session, server, sock_info, slave_ok) File "/Users/nauman.shahid/Nauman-data/Dev_Home/python/Venv/cyborg_lms_mvt/lib/python3.7/site-packages/pymongo/database.py", line 816, in _cmd kwargs) File "/Users/nauman.shahid/Nauman-data/Dev_Home/python/Venv/cyborg_lms_mvt/lib/python3.7/site-packages/pymongo/database.py", line 766, in _list_collections session=tmp_session)["cursor"] File "/Users/nauman.shahid/Nauman-data/Dev_Home/python/Venv/cyborg_lms_mvt/lib/python3.7/site-packages/pymongo/database.py", line 630, in _command client=self.__client) File "/Users/nauman.shahid/Nauman-data/Dev_Home/python/Venv/cyborg_lms_mvt/lib/python3.7/site-packages/pymongo/pool.py", line 613, in command user_fields=user_fields) File "/Users/nauman.shahid/Nauman-data/Dev_Home/python/Venv/cyborg_lms_mvt/lib/python3.7/site-packages/pymongo/network.py", line 167, in command parse_write_concern_error=parse_write_concern_error) File "/Users/nauman.shahid/Nauman-data/Dev_Home/python/Venv/cyborg_lms_mvt/lib/python3.7/site-packages/pymongo/helpers.py", line 159, in _check_command_response raise OperationFailure(msg % errmsg, code, response) pymongo.errors.OperationFailure: command listCollections requires authentication

[Naumansh]

yeah the downgrade worked for me too. thanks @ctow123

[nesdis]

the database configuration has changed with latest version. you now need to configure under 'client' https://nesdis.github.io/djongo/integrating-django-with-mongodb/#database-configuration

DATABASES = {
        'default': {
            'ENGINE': 'djongo',
            'ENFORCE_SCHEMA': True,
            'LOGGING': {
                'version': 1,
                'loggers': {
                    'djongo': {
                        'level': 'DEBUG',
                        'propogate': False,                        
                    }
                },
             },
            'NAME': 'your-db-name',
            'CLIENT': {
                'host': 'host-name or ip address',
                'port': port_number,
                'username': 'db-username',
                'password': 'password',
                'authSource': 'db-name',
                'authMechanism': 'SCRAM-SHA-1'
            }
        }
    }

[Daryhez]

the database configuration has changed with latest version. you now need to configure under 'client' https://nesdis.github.io/djongo/integrating-django-with-mongodb/#database-configuration

DATABASES = {
        'default': {
            'ENGINE': 'djongo',
            'ENFORCE_SCHEMA': True,
            'LOGGING': {
                'version': 1,
                'loggers': {
                    'djongo': {
                        'level': 'DEBUG',
                        'propogate': False,                        
                    }
                },
             },
            'NAME': 'your-db-name',
            'CLIENT': {
                'host': 'host-name or ip address',
                'port': port_number,
                'username': 'db-username',
                'password': 'password',
                'authSource': 'db-name',
                'authMechanism': 'SCRAM-SHA-1'
            }
        }
    }

It works for me! Thank you! So much!

[borinmin]

@Daryhez your codes are work for me on djongo==1.3.3 but not djongo==1.2.3. And for djongo==1.2.3, have to follow @ctow123 instructed to be worked. Thank all.

[Sykzen]

the database configuration has changed with latest version. you now need to configure under 'client' https://nesdis.github.io/djongo/integrating-django-with-mongodb/#database-configuration

DATABASES = {
        'default': {
            'ENGINE': 'djongo',
            'ENFORCE_SCHEMA': True,
            'LOGGING': {
                'version': 1,
                'loggers': {
                    'djongo': {
                        'level': 'DEBUG',
                        'propogate': False,                        
                    }
                },
             },
            'NAME': 'your-db-name',
            'CLIENT': {
                'host': 'host-name or ip address',
                'port': port_number,
                'username': 'db-username',
                'password': 'password',
                'authSource': 'db-name',
                'authMechanism': 'SCRAM-SHA-1'
            }
        }
    }

yey thnks

[joej]

It looks like this is still a thing ...any idea how to actually get "./manage.py makemigrations" to work? Tried the user=admin/role=root, tried all different 'default' settings (mentioned above). Always get:

  File "/shark/joe-src/MST-Data/shark-tank-ingest/indexer/venv/lib/python3.6/site-packages/pymongo/helpers.py", line 167, in _check_command_response
    raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: command listCollections requires authentication, full error: {'ok': 0.0, 'errmsg': 'command listCollections requires authentication', 'code': 13, 'codeName': 'Unauthorized'}

asgiref==3.4.1 dataclasses==0.8 Django==3.2.8 djongo==1.3.6 dnspython==2.1.0 pkg-resources==0.0.0 pymongo==3.12.1 pytz==2021.3 sqlparse==0.2.4 typing-extensions==3.10.0.2

EDIT

I tested it by hand. From the command line, the host port user password database work. I tested the listing of collections, output below:

$ mongosh -u muser -p mpassword --authenticationDatabase "admin" mongodb://mongohostname:27017
test> use tempdb
switched to db tempdb
tempdb> db.tempcollection.find();
[ { _id: ObjectId("6175e9f75b616a61a68cafaa"), temp: 'temp' } ]
tempdb>

[dmugtasimov]

It does not seem like a good design to put all arguments under CLIENT key while there is already Django standard way of providing credentials for a database. I suggest to keep HOST, PORT, USER and PASSWORD in on top level and use any extra arguments under CLIENT (like authSource). This would compatible with general Django pattern and also follow the principle of least astonishment ( https://en.wikipedia.org/wiki/Principle_of_least_astonishment )