ory / kratos

The most scalable and customizable identity server on the market. Replace your Homegrown, Auth0, Okta, Firebase with better UX and DX. Has all the tablestakes: Passkeys, Social Sign In, Multi-Factor Auth, SMS, SAML, TOTP, and more. Written in Go, cloud native, headless, API-first. Available as a service on Ory Network and for self-hosters.
https://www.ory.sh/?utm_source=github&utm_medium=banner&utm_campaign=kratos
Apache License 2.0
11.3k stars 964 forks source link

Identity property mismatch between `GET:/browser` and `POST:?flow=${id}` during registration #3486

Open mannie-exe opened 1 year ago

mannie-exe commented 1 year ago

Preflight checklist

Ory Network Project

No response

Describe the bug

TL;DR is that `POST:/self-service/registration?flow={id}` "validates" form data as <key> instead of traits.<key>, unsuccessfully, according to a 400 (click me to read more)

I'm trying to proxy any IDP request from the client via a Node.js server. Setting the right `content-type` and cookies is not a problem, and Kratos seems to recognize my requests. Doing a `POST` to the registration service URL (for the password method) fails. Here's an example HTTP `POST` body: ``` csrf_token=WKujP4sPg8%2BJu7pLqmBNupQu9Bu5c%2Fzwn8Yq32R6NQp9Zxu0Vb1NIoGNn%2Bqj4ZsW%2BctByKKvvwDBpu7lC7XIWA%3D%3D&traits.name=&password=&traits.date_of_birth=&traits.email=&method=password ``` ...which resolves in a `400` due to "missing traits".
Here's what the CLI says during POST: ```shell time=2023-09-08T21:59:59-07:00 level=info msg=started handling request http_request=map[headers:map[accept:application/json content-type:application/x-www-form-urlencoded cookie:REDACTED] host:REDACTED method:POST path:/self-service/registration query:REDACTED remote:192.168.88.1:63122 scheme:https] time=2023-09-08T21:59:59-07:00 level=info msg=Encountered self-service flow error. audience=audit error=map[message:I[#] S[#/required] missing properties: "name", "date_of_birth", "email" ] http_request=map[headers:map[accept:application/json content-type:application/x-www-form-urlencoded cookie:REDACTED] host:REDACTED method:POST path:/self-service/registration query:REDACTED remote:192.168.88.1:63122 scheme:https] registration_flow=&{628316ea-ab67-4a35-a84f-21c139c13114 browser 2023-09-09 05:13:44.74644 +0000 UTC 2023-09-09 04:58:44.74644 +0000 UTC [123 125] REDACTED/self-service/registration/browser 0xc000ffcaa0 2023-09-08 21:58:44.747977 -0700 -0700 2023-09-08 21:58:44.747977 -0700 -0700 OLWr4hf3jJnL4HefFWmFbwNgB+K9Yl4IAc8uIOiCmPVRviuMTumg2nEdRY1umDp1yx87u6M4r9wAZZQX955hIQ== 08ad6bc8-76c4-497a-a20f-40627ed9253c [123 125] [] } service_name=Ory Kratos service_version=v1.0.0 time=2023-09-08T21:59:59-07:00 level=info msg=completed handling request http_request=map[headers:map[accept:application/json content-type:application/x-www-form-urlencoded cookie:REDACTED] host:REDACTED method:POST path:/self-service/registration query:REDACTED remote:REDACTED scheme:https] http_response=map[headers:map[cache-control:private, no-cache, no-store, must-revalidate content-type:application/json; charset=utf-8 vary:Cookie] size:2568 status:400 text_status:Bad Request took:195.0886ms] ```


And here are the self-service responses before and after the registration request `POST -> kratos://self-service/registration?flow={id}`.
Before POST (after GET -> kratos://self-service/registration/browser): ```json { "id": "99a64fa5-e607-4dcd-9818-1c795ced3c0c", "type": "browser", "expires_at": "2023-09-09T07:11:14.2743837Z", "issued_at": "2023-09-09T06:56:14.2743837Z", "request_url": "/self-service/registration/browser", "ui": { "action": "/self-service/registration?flow=99a64fa5-e607-4dcd-9818-1c795ced3c0c", "method": "POST", "nodes": [ { "type": "input", "group": "default", "attributes": { "name": "csrf_token", "type": "hidden", "value": "JC6kHkmqFQ7bB7m7bEQQijHRa3U/xQhpZbLztYb9+I7ji8qchox/mxalaxfsTtuxUtKVnu9yUbx0Kd/wccFAsg==", "required": true, "disabled": false, "node_type": "input" }, "messages": [], "meta": {} }, { "type": "input", "group": "password", "attributes": { "name": "traits.name", "type": "text", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1070002, "text": "Display Name", "type": "info" } } }, { "type": "input", "group": "password", "attributes": { "name": "password", "type": "password", "required": true, "autocomplete": "new-password", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1070001, "text": "Password", "type": "info" } } }, { "type": "input", "group": "password", "attributes": { "name": "traits.date_of_birth", "type": "date", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1070002, "text": "Date of Birth", "type": "info" } } }, { "type": "input", "group": "password", "attributes": { "name": "traits.email", "type": "email", "autocomplete": "email", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1070002, "text": "E-Mail", "type": "info" } } }, { "type": "input", "group": "password", "attributes": { "name": "method", "type": "submit", "value": "password", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1040001, "text": "Sign up", "type": "info", "context": {} } } } ] } } ```
After POST: ```json { "id": "99a64fa5-e607-4dcd-9818-1c795ced3c0c", "type": "browser", "expires_at": "2023-09-09T07:11:14.2743837Z", "issued_at": "2023-09-09T06:56:14.2743837Z", "request_url": "/self-service/registration/browser", "ui": { "action": "/self-service/registration?flow=99a64fa5-e607-4dcd-9818-1c795ced3c0c", "method": "POST", "nodes": [ { "type": "input", "group": "default", "attributes": { "name": "name", "type": "text", "disabled": false, "node_type": "input" }, "messages": [ { "id": 4000002, "text": "Property name is missing.", "type": "error", "context": { "property": "name" } } ], "meta": {} }, { "type": "input", "group": "default", "attributes": { "name": "date_of_birth", "type": "text", "disabled": false, "node_type": "input" }, "messages": [ { "id": 4000002, "text": "Property date_of_birth is missing.", "type": "error", "context": { "property": "date_of_birth" } } ], "meta": {} }, { "type": "input", "group": "default", "attributes": { "name": "email", "type": "text", "disabled": false, "node_type": "input" }, "messages": [ { "id": 4000002, "text": "Property email is missing.", "type": "error", "context": { "property": "email" } } ], "meta": {} }, { "type": "input", "group": "default", "attributes": { "name": "csrf_token", "type": "hidden", "value": "OiTz3/DqJR1yckTCQ9rlHEZaSfthrQZyU9Y8eV/B97T9gZ1dP8xPiL/Qlm7D0C4nJVm3ELEaX6dCTRA8qP1PiA==", "required": true, "disabled": false, "node_type": "input" }, "messages": [], "meta": {} }, { "type": "input", "group": "password", "attributes": { "name": "traits.name", "type": "text", "value": "", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1070002, "text": "Display Name", "type": "info" } } }, { "type": "input", "group": "password", "attributes": { "name": "password", "type": "password", "required": true, "autocomplete": "new-password", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1070001, "text": "Password", "type": "info" } } }, { "type": "input", "group": "password", "attributes": { "name": "traits.date_of_birth", "type": "date", "value": "", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1070002, "text": "Date of Birth", "type": "info" } } }, { "type": "input", "group": "password", "attributes": { "name": "traits.email", "type": "email", "value": "", "autocomplete": "email", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1070002, "text": "E-Mail", "type": "info" } } }, { "type": "input", "group": "password", "attributes": { "name": "method", "type": "submit", "value": "password", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1040001, "text": "Sign up", "type": "info", "context": {} } } } ] } } ```


NOTE: Both are requests with `Content-Type`: `application/json`, and the `POST` specifically accepts `application/x-www-form-urlencoded; charset=utf-8` (I added the charset after something I noticed further down). Alongside `traits.name`, `traits.date_of_birth`, and `traits.email`, the `POST` response has `name`, `date_of_birth`, and `email` nodes as well. I'm not sure if this is the problem, but it's the only difference I can see -- the latter are the only nodes with validation error messages while the former are populated with the correct, url-encoded values given to the `POST` body. So I tried changing the `required` properties in my schema from `` to `traits.`.

Before `POST` looks the same.
After POST: ```json { "id": "0b862e5e-cdc5-477a-9103-d7f633bb41f2", "type": "browser", "expires_at": "2023-09-09T07:21:33.0681115Z", "issued_at": "2023-09-09T07:06:33.0681115Z", "request_url": "/self-service/registration/browser", "ui": { "action": "/self-service/registration?flow=0b862e5e-cdc5-477a-9103-d7f633bb41f2", "method": "POST", "nodes": [ { "type": "input", "group": "default", "attributes": { "name": "csrf_token", "type": "hidden", "value": "0nz90oNftFd94iHg4FpqpVjcBCtud9CaKRHh/BpSeaAV2ZNQTHnewrBA80xgUKGeO9/6wL7AiU84is257W7BnA==", "required": true, "disabled": false, "node_type": "input" }, "messages": [], "meta": {} }, { "type": "input", "group": "default", "attributes": { "name": "traits\\.name", "type": "text", "disabled": false, "node_type": "input" }, "messages": [ { "id": 4000002, "text": "Property traits.name is missing.", "type": "error", "context": { "property": "traits.name" } } ], "meta": {} }, { "type": "input", "group": "default", "attributes": { "name": "traits\\.date_of_birth", "type": "text", "disabled": false, "node_type": "input" }, "messages": [ { "id": 4000002, "text": "Property traits.date_of_birth is missing.", "type": "error", "context": { "property": "traits.date_of_birth" } } ], "meta": {} }, { "type": "input", "group": "default", "attributes": { "name": "traits\\.email", "type": "text", "disabled": false, "node_type": "input" }, "messages": [ { "id": 4000002, "text": "Property traits.email is missing.", "type": "error", "context": { "property": "traits.email" } } ], "meta": {} }, { "type": "input", "group": "password", "attributes": { "name": "traits.name", "type": "text", "value": "", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1070002, "text": "Display Name", "type": "info" } } }, { "type": "input", "group": "password", "attributes": { "name": "password", "type": "password", "required": true, "autocomplete": "new-password", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1070001, "text": "Password", "type": "info" } } }, { "type": "input", "group": "password", "attributes": { "name": "traits.date_of_birth", "type": "date", "value": "", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1070002, "text": "Date of Birth", "type": "info" } } }, { "type": "input", "group": "password", "attributes": { "name": "traits.email", "type": "email", "value": "", "autocomplete": "email", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1070002, "text": "E-Mail", "type": "info" } } }, { "type": "input", "group": "password", "attributes": { "name": "method", "type": "submit", "value": "password", "disabled": false, "node_type": "input" }, "messages": [], "meta": { "label": { "id": 1040001, "text": "Sign up", "type": "info", "context": {} } } } ] } } ```


Now all the missing keys in the response are named `traits\\.key`. Am I missing a character set configuration somewhere? Reading the [docs](https://www.ory.sh/docs/kratos/debug/troubleshooting), I see that there may be issues if trying to complete a flow from across browsers. Would I be running into that? I'm not sure how to check. I should mention I'm using the node HTTP2 client, and both Kratos and my proxy are on the same TLD -- served with TLS. My request looks something like: ```js idp_client.connection.request({ ":method": method, ":path": idp_url, Accept: accept_mime, // "application/json" ...data.headers, // content-type + cookie } ``` Is this a bug, or am I missing a step in the self-service process? ~~I did notice that `GET -> kratos://self-service/registration/browser` itself returns the user interface, so I never bother hitting `GET -> kratos://self-service/registration/flows=${id}`. Could that be an issue?~~ (Apparently [that's fine](https://www.ory.sh/docs/kratos/reference/api#tag/frontend/operation/updateRegistrationFlow) during requests that accept `application/json`.) Other than that, I get the same results from the Node.js client as I do from Postman. A `400` bad request with mentions of missing properties (either `name` or `traits\\.name` depending on my schema at that moment). If this is an issue covered in the docs, I'd be happy to close this issue ASAP. Otherwise, any help would be lovely.

P.S. My identity schema, for completeness' sake (traits.key change reverted): ```json { "$schema": "http://json-schema.org/draft-07/schema#", "title": "Player", "description": "User/player schema", "type": "object", "properties": { "traits": { "type": "object", "properties": { "name": { "title": "Display Name", "description": "A player's display name", "type": "string" }, "date_of_birth": { "title": "Date of Birth", "description": "A player's date of birth", "type": "string", "format": "date" }, "email": { "title": "E-Mail", "description": "A player's e-mail address", "type": "string", "format": "email", "ory.sh/kratos": { "credentials": { "password": { "identifier": true } }, "verification": { "via": "email" }, "recovery": { "via": "email" } }, "maxLength": 320 } } } }, "required": ["name", "date_of_birth", "email"], "additionalProperties": false } ```

Reproducing the bug

Short

  1. GET -> kratos://self-service/registration/browser
  2. POST -> kratos://self-service/registration?flow=${id}
  3. 400 - Bad Request; missing properties

Complete

  1. Client navigates to a page /
  2. Page triggers on load a request to GET -> /user/join
  3. Server requests GET -> kratos://self-service/registration/browser during /user/join
  4. Server gives the client any cookies and the registration form
  5. Client submits data to POST -> /user/join
  6. Server forwards data with cookie and body to POST -> kratos://self-service/registration?flow=${id}
  7. Server receives a 400 due to body missing required properties

Relevant log output

time=2023-09-08T21:59:59-07:00 level=info msg=started handling request

http_request=map[headers:map[accept:application/json content-type:application/x-www-form-urlencoded cookie:REDACTED] host:REDACTED method:POST path:/self-service/registration query:REDACTED remote:192.168.88.1:63122 scheme:https]

time=2023-09-08T21:59:59-07:00 level=info msg=Encountered self-service flow error. audience=audit error=map[message:I[#] S[#/required] missing properties: "name", "date_of_birth", "email" <stack trace omitted>]

http_request=map[headers:map[accept:application/json content-type:application/x-www-form-urlencoded cookie:REDACTED] host:REDACTED method:POST path:/self-service/registration query:REDACTED remote:192.168.88.1:63122 scheme:https]

registration_flow=&{628316ea-ab67-4a35-a84f-21c139c13114  <nil> browser 2023-09-09 05:13:44.74644 +0000 UTC 2023-09-09 04:58:44.74644 +0000 UTC [123 125] REDACTED/self-service/registration/browser    0xc000ffcaa0 2023-09-08 21:58:44.747977 -0700 -0700 2023-09-08 21:58:44.747977 -0700 -0700 OLWr4hf3jJnL4HefFWmFbwNgB+K9Yl4IAc8uIOiCmPVRviuMTumg2nEdRY1umDp1yx87u6M4r9wAZZQX955hIQ== 08ad6bc8-76c4-497a-a20f-40627ed9253c [123 125] [] }

service_name=Ory Kratos service_version=v1.0.0

time=2023-09-08T21:59:59-07:00 level=info msg=completed handling request 

http_request=map[headers:map[accept:application/json content-type:application/x-www-form-urlencoded cookie:REDACTED] host:REDACTED method:POST path:/self-service/registration query:REDACTED remote:REDACTED scheme:https]

http_response=map[headers:map[cache-control:private, no-cache, no-store, must-revalidate content-type:application/json; charset=utf-8 vary:Cookie] size:2568 status:400 text_status:Bad Request took:195.0886ms]

Relevant configuration

No response

Version

v1.0.0

On which operating system are you observing this issue?

Windows

In which environment are you deploying?

Other

Additional Context

No response

jonas-jonas commented 1 year ago

I am not entirely sure, I understand the issue, but let me try to clarify a few things:

The JSON coming from Kratos (that contains the ui fields) is made to be rendered as is, using some kind of translation layer, such as a library, to convert it to HTML, or native UI elements. So naturally, Kratos expects the fields that are then submitted as part of POST request to match those.

Couldn't you just add the traits. prefix on your server side before submission?

mannie-exe commented 1 year ago

Hey!

Thanks for taking a look.

I am not entirely sure, I understand the issue, but let me try to clarify a few things:

I understand the confusion, there's a lot to parse and there was a lot to write.

The JSON coming from Kratos (that contains the ui fields) is made to be rendered as is, using some kind of translation layer, such as a library, to convert it to HTML, or native UI elements. So naturally, Kratos expects the fields that are then submitted as part of POST request to match those.

Yes, absolutely. I'm using Handlebars at the moment to construct an HTML form from the parsed JSON response. The input HTML node attributes are passed pretty much as-is (except disabled, for eg.).

Couldn't you just add the traits. prefix on your server side before submission?

Since the UI is built from the JSON response, as-is, the fields have the "correct" name property — at least according to the GET response (i.e. traits.name and not name). You can see an example body sent with a POST request, from my server to Kratos, near the top of my post. The only fields without traits. are method and password which seems to be fine.

Once that data reaches Kratos, however, the data is "rejected" because Kratos expected name instead of traits.name for example. And the validation changes the UI node count from 6 to 9 (6 from GETting Kratos flow, and 3 more appended by validation errors).

In this case, would I have to instead strip the traits. prefix from all HTML field names?

I made sure to run kratos migrate sql in case there was some issue with my schema not being accepted. Is there something wrong with how I may have specified my required fields? I know that I'm at least using my schema from trying to make changes to it earlier, but I think there's something wrong with how I'm specifying required fields maybe?

Hope that clears up the issue, and let me know if there are more logs I can provide.

JoblersTune commented 8 months ago

I had a similar issue. If it helps anyone else I think the problem is where the required section is placed in the identity schema. I believe it should be nested under the traits object, not on the same level as the traits object. So the schema should rather look like this:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Player",
  "description": "User/player schema",
  "type": "object",
  "properties": {
    "traits": {
      "type": "object",
      "properties": {
        "name": {
          "title": "Display Name",
          "description": "A player's display name",
          "type": "string"
        },
        "date_of_birth": {
          "title": "Date of Birth",
          "description": "A player's date of birth",
          "type": "string",
          "format": "date"
        },
        "email": {
          "title": "E-Mail",
          "description": "A player's e-mail address",
          "type": "string",
          "format": "email",
          "ory.sh/kratos": {
            "credentials": {
              "password": {
                "identifier": true
              }
            },
            "verification": {
              "via": "email"
            },
            "recovery": {
              "via": "email"
            }
          },
          "maxLength": 320
        }
      },
      "required": ["name", "date_of_birth", "email"],
      "additionalProperties": false
    }
  }
}

Also, sending the post request with application/json seems to better handle the object nesting than using application/x-www-form-urlencoded.

christoph-kluge commented 2 months ago

I got the same issue. It's working fine as application/x-www-form-urlencoded POST from the browser against kratos frontend but it does not work with application/json from the server.

What feels a bit odd is that the email-trait is even prefilled with the email. 🤔

{
  "type": "input",
  "group": "default",
  "attributes": {
    "name": "traits.email",
    "type": "hidden",
    "value": "john.doe@example.net",
    "disabled": false,
    "node_type": "input"
  },
  "messages": [
    {
      "id": 4000002,
      "text": "Property email is missing.",
      "type": "error",
      "context": {
        "property": "email"
      }
    }
  ],
  "meta": {}
}

Additionally on the SettingsFlow has the same behavior but it's a even a bit more unintuitive:

  1. Changing proffile traits through the has the same behavior. application/json does not work, while application/x-www-form-urlencoded works fine.
  2. Refreshing the page with the same flowId does not have any prefilled trait values. On the other hand a new settings-flow does show them again.
christoph-kluge commented 2 months ago

Solved for me: The payload must deserialize any dot-notation fields into object notation. After that everything from the api reference starts working for me. Example:


POST /self-service/registration?flow=${id}
Content-Type: application/json

- {
-   'traits.email': "john.doe@example.net",
-   'traits.name.first': "John",
-   'traits.name.last': "Doe"
- }
+ {
+   traits: {
+     email: "john.doe@example.net"
+     name: {
+       first: "John",
+       last: "Doe"
+     }
+   }
+ }
}