search

jayvynl / django-clickhouse-backend

Django clickhouse database backend.
MIT License
130 stars 21 forks source link

Using OAuth throws an exception on first login #25

Closed gsaenger closed 1 year ago

gsaenger commented 1 year ago

I am using OAuth to authenticate the user. On the first login of any user I get an Exception:

"ValueError at /oauth2/callback User: testuser.xxxxx@setango.onmicrosoft.com needs to have a value for field "id" before this many-to-many relationship can be used."

Request Method: GET Request URL: http://localhost:8000/oauth2/callback?code=0.Aa8AyJqK8agTxUGeKQfNkaLngKyK9yFuRJlLowdUuQ9I7zOvAAA.AgABAAIAAAD--DLA3VO7QrddgJg7WevrAgDs_wUA9P-Q7GuTyy8frIzyqyW92Mp9vr-fkoVb-0PTvBG0gIC1pcWPLSdjvNxeFSRykIW2CDlOjH3hHUha20WZ-6J9BZVwFwLVrksxT1N27gjGEy8Q4aVjfjX_OwJEbJVD0ZNePRVWiquxgTzaa10t3hT6X0SeVp_sQcktyHoLgJnjlOvq2zL0Cp01L_FrS2lrDvZzA0rviS8yrh6z2hmB2imoSGtEOMOu3zXNoDbjfAdMPQb3K2mAcK129X0D78C_uSEJ2t6tpVxqOnuQA7v_NSpMPvSWEgQX5pR7UtvkU_vzf2AMzWa-S3PxBs2GD2nJ8WPuxR-XY6lADvhHdWm82OVgiQuCmojMjkzvYpk7EXZr7f9Q1tlxE9zC6SxnDcBUr1GQZ29KBAPlYA7CcEnqxuKL99bgGU55B7Vx3G5crGJKK80oXrMnyuWG4UrjLipzHmzWMqa75ppqEC-COJy8DpaLeQkF548scn2fr5_I_w0csyFUCqRSadJXllGafZDA2HuOOe1ozXQztu8kanojALpzNoOQlPbdoDTUnVR3ybjqxd5d2RgIJqQBIn3tgLnaLNRV_r15XoxdJKAlUJ2H5QV8qp1xTRnDXTyU&state=L2luZGV4Lmh0bWw%3d&session_state=778f99e7-882e-47f7-a107-281cb2c2260a Django Version: 4.1.1 Exception Type: ValueError Exception Value:

"<User: testuser.xxxxx@setango.onmicrosoft.com>" needs to have a value for field "id" before this many-to-many relationship can be used.

Exception Location: /home/xxxxx/env/lib64/python3.11/site-packages/django/db/models/fields/related_descriptors.py, line 979, in init Raised during: django_auth_adfs.views.OAuth2CallbackView Python Executable: /home/xxxxx/env/bin/python3 Python Version: 3.11.4 Python Path:

['/home/xxx/work/repos/project/app', '/usr/lib64/python311.zip', '/usr/lib64/python3.11', '/usr/lib64/python3.11/lib-dynload', '/home/xxx/env/lib64/python3.11/site-packages', '/home/xxx/work/repos/project/tests', '/home/xxx/work/repos/project/pymodules', '/home/xxx/env/lib/python3.11/site-packages']

On Django console:

Connected to ClickHouse server version 23.5.3, revision: 54462 Query: SELECT "auth_user"."id", "auth_user"."password", "auth_user"."last_login", "auth_user"."is_superuser", "auth_user"."username", "auth_user"."first_name", "auth_user"."last_name", "auth_user"."email", "auth_user"."is_staff", "auth_user"."is_active", "auth_user"."date_joined" FROM "auth_user" WHERE "auth_user"."username" = 'testuser.xxxxx@setango.onmicrosoft.com' LIMIT 21 Block "" send time: 0.000024 Query: INSERT INTO "auth_user"("password", "last_login", "is_superuser", "username", "first_name", "last_name", "email", "is_staff", "is_active", "date_joined") VALUES Block "" send time: 0.000030 Block "" send time: 0.000224 Block "" send time: 0.000032 User 'testuser.xxxxx@setango.onmicrosoft.com' has been created. Attribute 'first_name' for instance 'testuser.xxxxx@setango.onmicrosoft.com' was set to 'Test'. Attribute 'last_name' for instance 'testuser.xxxxx@setango.onmicrosoft.com' was set to 'User E2E'. Attribute 'email' for instance 'testuser.xxxxx@setango.onmicrosoft.com' was set to 'testuser.xxxxx@setango.onmicrosoft.com'. Internal Server Error: /oauth2/callback ....

But the entry seem somehow to be generated, if I go back to the start page, everything works fine, the user is authenticated and can access the site. Login/logout works fine, no issues...

If a second user logs in the same happens. Afterwards I get a second problem (not sure if related to the missing id, that's why I put it In here):

get() returned more than one User -- it returned 2!

Request Method: GET Request URL: http://localhost:8000/oauth2/callback?code=0.Aa8AyJqK8agTxUGeKQfNkaLngKyK9yFuRJlLowdUuQ9I7zOvAAA.AgABAAIAAAD--DLA3VO7QrddgJg7WevrAgDs_wUA9P8uM_O2KwxsGRB2rqMqALBcUu1_IlMBqqmTqBCiuP1wbmVCTyMrWUK5-chrWgWyAhTFGPa4wgKCvnlSx6A_uUDEG1MvLsTQwjFH2TP3dWgIfWZDdV2AwLGk_L20BepiAkSVB_TtE9tY3vBU30PPFug18jL9hyWSgoZPBvoHuCaECr_ANRBUnZjHmSf6PgW1bqT05LQixHwp7WRcbxYTNdRa_WjDt-ts47iuMlO3hX2lZ18spPF2VSmfZjp9poJrjSANWuoe7Do_oSsUNi70qDzcltqvrn1IhLN8YHYBFwg1SaNgfEJiQfj5z0lubymID1Dd6wyiq2qclm0ycfoW2E7Gii2B3W81oSRr5KEh9JDkKMa3D9qHGjzbZDwr7m3E8-GQ8ZCH2OdUTnJWagMi1nQYRnmOSnNx_MSjSQIRUHMaf_l_Jk7XdNCZ0tzzOxMGGiwR5DIIh0nqT28BGpnb50mUYehh8ETzvcWY51BqF1QPl8rLIFUb7kqzKF156Xi_W7FJP3Rgnm9qAkBvYDp40x8Qebzjs0egPTLFcPJYMIvf9ED1VKeJrA2i4n4MlqTvmF2sKd02eSyMekSUWOsZagKgNdEZwrdXmw&state=L2luZGV4Lmh0bWw%3d&session_state=5860aa97-abc1-4b8e-9764-3b101f4fadd9 Django Version: 4.1.1 Exception Type: MultipleObjectsReturned Exception Value:

get() returned more than one User -- it returned 2!

Exception Location: /home/xxx/env/lib64/python3.11/site-packages/django/db/models/query.py, line 653, in get Raised during: django_auth_adfs.views.OAuth2CallbackView Python Executable: /home/xxx/env/bin/python3 Python Version: 3.11.4 Python Path:

['/home/xxx/work/repos/projekt/app', '/usr/lib64/python311.zip', '/usr/lib64/python3.11', '/usr/lib64/python3.11/lib-dynload', '/home/xxx/env/lib64/python3.11/site-packages', '/home/xxx/work/repos/projekt/tests', '/home/xxx/work/repos/projekt/pymodules', '/home/xxx/env/lib/python3.11/site-packages']

Server time: Tue, 18 Jul 2023 11:56:54 +0000

In the database I see two identical (!) entries, same username, same timestamp...

gsaenger commented 1 year ago

It seems that the id is at the first insert not set and therefor "0", a subsequent request is then applied to all entries in the CH? Usually (in a mariadb or sqlite) the id column is an auto-increment. CH does not support this and relies on the app. A solution might be using something different as id like a checksum over the object-id (from ad) or similar?

jayvynl commented 1 year ago

have you tried adding DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField' in settings?

gsaenger commented 1 year ago

It is in...

jayvynl commented 1 year ago

It is in...

Django settings, file path is usually {project_name}/settings.py

gsaenger commented 1 year ago

Sorry, was ambiguous. It is in the settings file already.

jayvynl commented 1 year ago

Thanks for the report.

Clickhouse lacks auto increamental type, foreignkey and transaction. So many existing apps may not work.

I will try to fix this. But for now, you can store your user app in RDBMS, and store other time series data in clickhouse. Clickhouse is good at OLAP, but not OLTP.

gsaenger commented 1 year ago

Thanks a lot! If you need something debugged or tested, let me know...

jayvynl commented 1 year ago

This bug is partially fixed by #31. Now id worker will generate id value for user instance.

But there are still some limitations.