Unable to connect to database, failed to parse as URL

AurimasNav commented 2 years ago

Preflight checklist

[X] I could not find a solution in the existing issues, docs, nor discussions.
[X] I agree to follow this project's Code of Conduct.
[X] I have read and am following this repository's Contribution Guidelines.
[ ] This issue affects my Ory Cloud project.
[ ] I have joined the Ory Community Slack.
[ ] I am signed up to the Ory Security Patch Newsletter.

Describe the bug

automigrate container fails due to failed parsing of postgres address as URL.

Reproducing the bug

configure dsn for postgres on azure in helm values with password including symbols #!$%&+{}<>:
try to deploy with helm

Relevant log output

> time=2022-03-08T08:32:42Z level=warning msg=Unable to connect to database, retrying. audience=application error=map[message:cannot parse `postgres:xxxxxx@dev-psql:xxxxxx@dev-psql.postgres.database.azure.com:5432/orykratos-dev-psqldb?sslmode=verify-full`: failed to parse as URL (parse "postgres://**user**:**password_fragment**": invalid port ":**password_fragment**" after host)] service_name=Ory Kratos service_version=v0.8.0-alpha.3

Relevant configuration

kratos:
  config:
    dsn: postgress://user:password_with_special_symbols_#!&+{}<>:@postgres_fqdn:port/db_name_etc.

Version

v0.8.0-alpha.3

On which operating system are you observing this issue?

Linux

In which environment are you deploying?

Kubernetes with Helm

Additional Context

Not sure if qualifies as a bug, if not then it would be nice if in the configuration docs it would be mentioned that symbols in dsn need to be percent-encoded for url parsing.

aeneasr commented 2 years ago

Try url-encoding it, so password_with_special_symbols_%23%21%26%2B%7B%7D%3C%3E%3A, see https://stackoverflow.com/questions/6718471/escaping-username-characters-in-basic-auth-urls

imchasingshadows commented 2 years ago

I think kratos/keto should provide a mechanism that allows for providing database credentials (passwords) with special characters.

Our usecase is that we run Kratos and keto in AWS ECS which provides databasae credentials via a secret stored in secret manager. This password is autogenerated and can possibly contain illegal url characters.

I solved this by adding an entrypoint script which encodes the password before sending it to kratos/keto. but imo the encoding should be handled by kratos/keto. I'd prefer to specify database name, password, username etc. as seperate env vars.

#!/bin/sh
url_encode() (
    string=${*:-$(
        cat -
        printf x
    )}
    [ -n "$*" ] || string=${string%x}
    # Zero index, + 1 to start from 1 since sed starts from 1
    lines=$(($(printf %s "$string" | wc -l) + 1))
    lineno=1
    while [ $lineno -le $lines ]; do
        currline=$(printf %s "$string" | sed "${lineno}q;d")
        pos=1
        chars=$(printf %s "$currline" | wc -c)
        while [ $pos -le "$chars" ]; do
            c=$(printf %s "$currline" | cut -b$pos)
            case $c in
            [-_.~a-zA-Z0-9]) printf %c "$c" ;;
            *) printf %%%02X "'${c:-\n}'" ;;
            esac
            pos=$((pos + 1))
        done
        [ $lineno -eq $lines ] || printf %%0A
        lineno=$((lineno + 1))
    done
)

DSN="postgres://$DATABASE_USERNAME:$(url_encode "$DATABASE_PASSWORD")@$DATABASE_HOST:$DATABASE_PORT/$DATABASE_NAME?sslmode=verify-full&sslrootcert=$(url_encode "/home/ory/.postgresql/root.crt")"
export DSN
/usr/bin/keto "$@"

ory / kratos