Flow sometimes gives ErrorValidationGeneric with text matching more precise types

meyfa commented 2 years ago

Preflight checklist

[X] I could not find a solution in the existing issues, docs, nor discussions.
[X] I agree to follow this project's Code of Conduct.
[X] I have read and am following this repository's Contribution Guidelines.
[ ] This issue affects my Ory Cloud project.
[ ] I have joined the Ory Community Slack.
[ ] I am signed up to the Ory Security Patch Newsletter.

Describe the bug

Sending a request with empty or invalid fields will return UI messages with code 4000001 (ErrorValidationGeneric) but the message text itself contains a more precise error such as "length must be >= 3, but got 1" or "a" is not valid "email". These texts seem to come from message codes 4000003 (ErrorValidationMinLength) and 4000004 (ErrorValidationInvalidFormat), respectively.

To summarize the problem with this:

Code 4000001 is given for many different kinds of messages
Its text depends on user input
The context property is missing

This makes it impossible to localize the message client-side.

Reproducing the bug

Try performing a registration with an invalid email address:

POST /self-service/registration?flow=370b24b4-fe63-4e58-ad27-b8316c7467d5 HTTP/1.1

{
  "csrf_token":"E/eJz9dRUlGEk6ySEsMq23mAbTZnIW79rb1rUPYPuDy4IpXFVzLFum9IIl5/PqAJIrx73UTD8/shwAEoXPQ0xQ==",
  "traits.email":"a",
  "password":"aTV0fKzPGITmVGYx+QI5DqfDkj/S4HTebRb+Eu4dpwU=",
  "traits.name":"a",
  "method":"password"
}

Then the response body looks something like:

{
  "id": "370b24b4-fe63-4e58-ad27-b8316c7467d5",
  "type": "browser",
  "expires_at": "2022-01-04T11:39:30.453402Z",
  "issued_at": "2022-01-04T10:39:30.453402Z",
  "request_url": "http://localhost:8080/self-service/registration/browser",
  "ui": {
    "action": "http://localhost:3000/ory/self-service/registration?flow=370b24b4-fe63-4e58-ad27-b8316c7467d5",
    "method": "POST",
    "nodes": [
      {
        "type": "input",
        "group": "default",
        "attributes": {
          "name": "csrf_token",
          "type": "hidden",
          "value": "ldw8+7jugMwpAfw43+4bmxIh/WZaYAxhdstCD31Kdhg+CSDxOI0XJ8LacvSyE5FJSR3rjXmCkWf6tih317H64Q==",
          "required": true,
          "disabled": false,
          "node_type": "input"
        },
        "messages": [],
        "meta": {}
      },
      {
        "type": "input",
        "group": "password",
        "attributes": {
          "name": "traits.email",
          "type": "email",
          "value": "a",
          "required": true,
          "disabled": false,
          "node_type": "input"
        },
        "messages": [
          {
            "id": 4000001,
            "text": "\"a\" is not valid \"email\"",
            "type": "error"
          },
          {
            "id": 4000001,
            "text": "length must be \\u003e= 3, but got 1",
            "type": "error"
          }
        ],
        "meta": {
          "label": {
            "id": 1070002,
            "text": "E-Mail",
            "type": "info"
          }
        }
      },
      {
        "type": "input",
        "group": "password",
        "attributes": {
          "name": "password",
          "type": "password",
          "required": true,
          "disabled": false,
          "node_type": "input"
        },
        "messages": [],
        "meta": {
          "label": {
            "id": 1070001,
            "text": "Password",
            "type": "info"
          }
        }
      },
      {
        "type": "input",
        "group": "password",
        "attributes": {
          "name": "traits.name",
          "type": "text",
          "value": "a",
          "required": true,
          "disabled": false,
          "node_type": "input"
        },
        "messages": [],
        "meta": {
          "label": {
            "id": 1070002,
            "text": "Full name",
            "type": "info"
          }
        }
      },
      {
        "type": "input",
        "group": "password",
        "attributes": {
          "name": "method",
          "type": "submit",
          "value": "password",
          "disabled": false,
          "node_type": "input"
        },
        "messages": [],
        "meta": {
          "label": {
            "id": 1040001,
            "text": "Sign up",
            "type": "info",
            "context": {}
          }
        }
      }
    ]
  }
}

Notice the messages under traits.email.

Relevant log output

time=2022-01-04T10:56:17Z level=info msg=started handling request http_request=map[headers:map[accept:application/json, text/plain, */* accept-encoding:gzip, deflate accept-language:en-US,en;q=0.8,de-DE;q=0.5,de;q=0.3 connection:close content-length:220 content-type:application/json cookie:Value is sensitive and has been redacted. To see the value set config key "log.leak_sensitive_values = true" or environment variable "LOG_LEAK_SENSITIVE_VALUES=true". dnt:1 origin:http://localhost:3000 referer:http://localhost:3000/register sec-fetch-dest:empty sec-fetch-mode:cors sec-fetch-site:same-origin sec-gpc:1 user-agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:95.0) Gecko/20100101 Firefox/95.0] host:localhost:8080 method:POST path:/self-service/registration query:Value is sensitive and has been redacted. To see the value set config key "log.leak_sensitive_values = true" or environment variable "LOG_LEAK_SENSITIVE_VALUES=true". remote:10.244.0.1:39351 scheme:http]
time=2022-01-04T10:56:17Z level=info msg=Encountered self-service flow error. audience=audit error=map[message:I[#/traits] S[#/properties/traits/$ref] doesn't validate with "base64://ewogICIkaWQiOiAiaHR0cHM6Ly9zY2hlbWFzLm9yeS5zaC9wcmVzZXRzL2tyYXRvcy9xdWlja3N0YXJ0L2VtYWlsLXBhc3N3b3JkL2lkZW50aXR5LnNjaGVtYS5qc29uIiwKICAiJHNjaGVtYSI6ICJodHRwOi8vanNvbi1zY2hlbWEub3JnL2RyYWZ0LTA3L3NjaGVtYSMiLAogICJ0aXRsZSI6ICJQZXJzb24iLAogICJ0eXBlIjogIm9iamVjdCIsCiAgInByb3BlcnRpZXMiOiB7CiAgICAidHJhaXRzIjogewogICAgICAidHlwZSI6ICJvYmplY3QiLAogICAgICAicHJvcGVydGllcyI6IHsKICAgICAgICAiZW1haWwiOiB7CiAgICAgICAgICAidHlwZSI6ICJzdHJpbmciLAogICAgICAgICAgImZvcm1hdCI6ICJlbWFpbCIsCiAgICAgICAgICAidGl0bGUiOiAiRS1NYWlsIiwKICAgICAgICAgICJtaW5MZW5ndGgiOiAzLAogICAgICAgICAgIm9yeS5zaC9rcmF0b3MiOiB7CiAgICAgICAgICAgICJjcmVkZW50aWFscyI6IHsKICAgICAgICAgICAgICAicGFzc3dvcmQiOiB7CiAgICAgICAgICAgICAgICAiaWRlbnRpZmllciI6IHRydWUKICAgICAgICAgICAgICB9CiAgICAgICAgICAgIH0sCiAgICAgICAgICAgICJ2ZXJpZmljYXRpb24iOiB7CiAgICAgICAgICAgICAgInZpYSI6ICJlbWFpbCIKICAgICAgICAgICAgfSwKICAgICAgICAgICAgInJlY292ZXJ5IjogewogICAgICAgICAgICAgICJ2aWEiOiAiZW1haWwiCiAgICAgICAgICAgIH0KICAgICAgICAgIH0KICAgICAgICB9LAogICAgICAgICJuYW1lIjogewogICAgICAgICAgInR5cGUiOiAic3RyaW5nIiwKICAgICAgICAgICJ0aXRsZSI6ICJGdWxsIG5hbWUiCiAgICAgICAgfQogICAgICB9LAogICAgICAicmVxdWlyZWQiOiBbCiAgICAgICAgImVtYWlsIiwKICAgICAgICAibmFtZSIKICAgICAgXSwKICAgICAgImFkZGl0aW9uYWxQcm9wZXJ0aWVzIjogZmFsc2UKICAgIH0KICB9Cn0K#/properties/traits"
  I[#/traits/email] S[#/properties/traits/properties/email] validation failed
    I[#/traits/email] S[#/properties/traits/properties/email/format] "a" is not valid "email"
    I[#/traits/email] S[#/properties/traits/properties/email/minLength] length must be >= 3, but got 1] http_request=map[headers:map[accept:application/json, text/plain, */* accept-encoding:gzip, deflate accept-language:en-US,en;q=0.8,de-DE;q=0.5,de;q=0.3 connection:close content-length:220 content-type:application/json cookie:Value is sensitive and has been redacted. To see the value set config key "log.leak_sensitive_values = true" or environment variable "LOG_LEAK_SENSITIVE_VALUES=true". dnt:1 origin:http://localhost:3000 referer:http://localhost:3000/register sec-fetch-dest:empty sec-fetch-mode:cors sec-fetch-site:same-origin sec-gpc:1 user-agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:95.0) Gecko/20100101 Firefox/95.0] host:localhost:8080 method:POST path:/self-service/registration query:Value is sensitive and has been redacted. To see the value set config key "log.leak_sensitive_values = true" or environment variable "LOG_LEAK_SENSITIVE_VALUES=true". remote:10.244.0.1:39351 scheme:http] registration_flow=&{370b24b4-fe63-4e58-ad27-b8316c7467d5 browser 2022-01-04 11:39:30.453402 +0000 UTC 2022-01-04 10:39:30.453402 +0000 UTC [123 125] http://localhost:8080/self-service/registration/browser   0xc000d96e60 2022-01-04 10:39:30.454911 +0000 UTC 2022-01-04 10:39:30.454911 +0000 UTC L4biI2ar+eiuN581sFdOKm3WSU6yflA75FMJRS5gzrOEU/4p5shuA0XsEfndqsT4NupfpZGczT1oLmM9hJtCSg== d72aaa0b-0f50-4935-b85f-bbc0aecb51ac} service_name=Ory Kratos service_version=v0.8.0-alpha.3
time=2022-01-04T10:56:17Z level=info msg=completed handling request http_request=map[headers:map[accept:application/json, text/plain, */* accept-encoding:gzip, deflate accept-language:en-US,en;q=0.8,de-DE;q=0.5,de;q=0.3 connection:close content-length:220 content-type:application/json cookie:Value is sensitive and has been redacted. To see the value set config key "log.leak_sensitive_values = true" or environment variable "LOG_LEAK_SENSITIVE_VALUES=true". dnt:1 origin:http://localhost:3000 referer:http://localhost:3000/register sec-fetch-dest:empty sec-fetch-mode:cors sec-fetch-site:same-origin sec-gpc:1 user-agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:95.0) Gecko/20100101 Firefox/95.0] host:localhost:8080 method:POST path:/self-service/registration query:Value is sensitive and has been redacted. To see the value set config key "log.leak_sensitive_values = true" or environment variable "LOG_LEAK_SENSITIVE_VALUES=true". remote:10.244.0.1:39351 scheme:http] http_response=map[headers:map[cache-control:private, no-cache, no-store, must-revalidate content-type:application/json; charset=utf-8 vary:Cookie] size:1668 status:400 text_status:Bad Request took:5.620665ms]

Relevant configuration

{
  "$id": "https://schemas.ory.sh/presets/kratos/quickstart/email-password/identity.schema.json",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Person",
  "type": "object",
  "properties": {
    "traits": {
      "type": "object",
      "properties": {
        "email": {
          "type": "string",
          "format": "email",
          "title": "E-Mail",
          "minLength": 3,
          "ory.sh/kratos": {
            "credentials": {
              "password": {
                "identifier": true
              }
            },
            "verification": {
              "via": "email"
            },
            "recovery": {
              "via": "email"
            }
          }
        },
        "name": {
          "type": "string",
          "title": "Full name"
        }
      },
      "required": [
        "email",
        "name"
      ],
      "additionalProperties": false
    }
  }
}

Version

v0.8.0-alpha.3

On which operating system are you observing this issue?

Linux

In which environment are you deploying?

Kubernetes with Helm

Additional Context

This was previously reported in #1241 which was closed due to inactivity, although the problem remains.

I suspect the problem lies in https://github.com/ory/kratos/blob/0fe4155b878102b77f7f13de5f0754ff75961498/ui/container/container.go#L181 where the original error type is erased. Unfortunately I haven't got a clue on how this could/should be fixed.

aeneasr commented 2 years ago

Great points! The localization part is still very rough at the moment. The problem is that the error messages are coming from several sources

Errors
JSON Schema Validation Messages & Errors
Hardcoded conditionals (such as password length)

and since all of them use somewhat different formats it is a bit difficult to homogenize them.

However, this should be our target goal. One opportunity for improvement is to merge https://github.com/ory/jsonschema with the most recent version (v5) of https://github.com/santhosh-tekuri/jsonschema as I think they/he made a lot of improvements to the validation error handling.

I do think though that this will be a bit of work as we had to make quite a lot of adjustments in the code base to support custom hooks & better validation messages.

Any help would be appreciated here. It's basically grunt work... If you're interested I can lay out a plan what needs to be done to merge the two

meyfa commented 2 years ago

Sure, why not. I'll see what I can do.

meyfa commented 2 years ago

@aeneasr May I ask, what process did you have in mind for merging the large number of changes from upstream? (I would ask directly on ory/jsonschema but there is no Issues tab there.)

aeneasr commented 2 years ago

Sorry, I missed this - no idea to be honest. Maybe making a diff with the latest v2 version and ory/jsonschema, and then a new branch off of the next version, and then looking at the diff and trying to apply it.

That or resolving the conflicts using git merge (probably easier) / rebase

github-actions[bot] commented 1 month ago

Hello contributors!

I am marking this issue as stale as it has not received any engagement from the community or maintainers for a year. That does not imply that the issue has no merit! If you feel strongly about this issue

open a PR referencing and resolving the issue;
leave a comment on it and discuss ideas on how you could contribute towards resolving it;
leave a comment and describe in detail why this issue is critical for your use case;
open a new issue with updated details and a plan for resolving the issue.

Throughout its lifetime, Ory has received over 10.000 issues and PRs. To sustain that growth, we need to prioritize and focus on issues that are important to the community. A good indication of importance, and thus priority, is activity on a topic.

Unfortunately, burnout has become a topic of concern amongst open-source projects.

It can lead to severe personal and health issues as well as opening catastrophic attack vectors.

The motivation for this automation is to help prioritize issues in the backlog and not ignore, reject, or belittle anyone.

If this issue was marked as stale erroneously you can exempt it by adding the backlog label, assigning someone, or setting a milestone for it.

Thank you for your understanding and to anyone who participated in the conversation! And as written above, please do participate in the conversation if this topic is important to you!

Thank you 🙏✌️

ory / kratos