Implementation of Kratos + Hydra have problems.

[DeamonMV]

Hello atreya2011

First of all, thank you for your work, it's very nice to have such example.

I tried your implementation Kratos + Hydra, from branch hydra, but I faced two problem, and for one I have solution.

My ENV:

OS Debian 10 Docker 20.03 I'm running here your implementation and then from other PC reaching this resource.

Problems

1) Problem Hydra do not starting.

{"audience":"application","error":{"message":"ERROR: relation \"hydra_jwk\" does not exist (SQLSTATE 42P01)"},"level":"fatal","msg":"Could not ensure that signing keys for \"hydra.openid.id-token\" exists. If you are running against a persistent SQL database this is most likely because your \"secrets.system\" (\"SECRETS_SYSTEM\" environment variable) is not set or changed. When running with an SQL database backend you need to make sure that the secret is set and stays the same, unless when doing key rotation. This may also happen when you forget to run \"hydra migrate sql\"..","service_name":"ORY Hydra","service_version":"v1.10.6","time":"2021-11-28T16:49:37Z"}

I have new ENV SECRETS_SYSTEM to docker-compose.yml file, like this, for hydra-migrate and hydra

environment:
      - SECRETS_SYSTEM=some_test_secret_change_me

length of this secret must be >16

2) Login end with an error

[PUT /oauth2/auth/requests/login/accept][400] acceptLoginRequestBadRequest  &{Error:invalid_request ErrorDebug: ErrorDescription:The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed. Query parameter 'challenge' is not defined but should have been. StatusCode:0}

How to repeat:

• create a new user
• login
• get an error

For this one I can't say what need todo to fix it.

Can you please help with it?

Thank you.

[atreya2011]

Hi @DeamonMV

Thank you and I am glad the implementation is useful to you 🙇🏼‍♂️

Please try the following after checking out the hydra-consent branch by executing: git checkout hydra-consent

Problem Hydra do not starting.

Unfortunately, this is an issue I am not able to resolve yet. The first time you execute docker-compose up --build, you should be getting that error. Can you try docker-compose down and docker-compose up --build again? That should get it working :)

Login end with an error

This is another issue I have to solve. Can you try the following and let me know if it works for you?

1. After creating a new user, open http://localhost:4436 and click on the verification link to verify the newly created user
2. Open http://localhost:4455/logout to log out and clear cookies

Then do the following:

1. Open a new terminal session, and create an OAuth2 Client

docker-compose exec hydra \
    hydra clients create \
    --endpoint http://127.0.0.1:4445 \
    --id auth-code-client \
    --secret secret \
    --grant-types authorization_code,refresh_token \
    --response-types code,id_token \
    --scope openid,offline \
    --callbacks http://127.0.0.1:5555/callback

2. Then, execute the following to start the OAuth2 Authorize Code Flow

docker-compose exec hydra \
    hydra token user \
    --client-id auth-code-client \
    --client-secret secret \
    --endpoint http://127.0.0.1:4444/ \
    --port 5555 \
    --scope openid,offline

3. Open http://localhost:5555 in your browser and follow the on-screen instructions to log in.

[DeamonMV]

Problem with start hydra container

So, I checked out to the hydra-consent branch and tried this:

• I run docker-compose up --build - here I got error about (\"SECRETS_SYSTEM\" environment variable) is not set or changed
• then I did 'docker-compose down`
• then docker-compose up --build again. But unfortunately error was still present.

I tried add SECRETS_SYSTEM env again - all container become up and running.

Note: While I check if container is up and running in logs of hydra container I noticed this two lines

{"audience":"application","level":"warning","msg":"JSON Web Key Set \"hydra.openid.id-token\" does not exist yet, generating new key pair...","service_name":"ORY Hydra","service_version":"v1.10.6","time":"2021-12-01T12:42:55Z"}
{"audience":"application","level":"warning","msg":"JSON Web Key Set \"hydra.jwt.access-token\" does not exist yet, generating new key pair...","service_name":"ORY Hydra","service_version":"v1.10.6","time":"2021-12-01T12:42:57Z"}

As I understand two exec command is solving this problem

Problem with login

Tried in two ways:

1) everything running on docker host, everywhere I changed from 127.0.0.1 to 192.168.0.10, where 192.168.0.10 that is IP of my docker host.

when I run this command

 docker-compose exec hydra \
>     hydra token user \
>     --client-id auth-code-client \
>     --client-secret secret \
>     --endpoint http://127.0.0.1:4444/ \
>     --port 5555 \
>     --scope openid,offline

I got this result

Setting up home route on http://127.0.0.1:5555/
Setting up callback listener on http://127.0.0.1:5555/callback
Press ctrl + c on Linux / Windows or cmd + c on OSX to end the process.
If your browser does not open automatically, navigate to:

        http://127.0.0.1:5555/

Apparently I missed something, because of IP is still 127.0.0.1

And on URL - http://192.168.0.10:5555 nothing happened

2) I run without changing 127.0.0.1 to 192.168.0.10.

After I opened http://127.0.0.1:5555/ and go through instructions I got Access Token: Refresh Token: and other.

But when I'm trying to login http://127.0.0.1:4455/login it's ends with the same error.

PS

Also I noticed that I need to restarting container with hydra migration and hydra it self, in order to have them up and running. As I understand docker starting containers to fast and hydra migrations can't apply migration, because something still missed.

[atreya2011]

Thanks for the detailed report! This is really helpful for me to debug on my side.

Let me try to reproduce the error you are facing and maybe I will be able to resolve the error of migrations not running properly on starting up the container initially...

Also, sorry, but I noticed an error in my instructions.

Once you check out the hydra-consent branch, all URLs should be 127.0.0.1 and not localhost.

Thank you for pointing this out 🙏🏼

After I opened http://127.0.0.1:5555/ and go through instructions I got Access Token: Refresh Token: and other.

This is great! So once you get your access token and refresh token, you need to run this command again, and go through the OAuth2 Authorization Flow.

docker-compose exec hydra \
    hydra token user \
    --client-id auth-code-client \
    --client-secret secret \
    --endpoint http://127.0.0.1:4444/ \
    --port 5555 \
    --scope openid,offline

if you try to open the 127.0.0.1:4455/login without going through the flow, then you will get the query parameter challenge is not defined as you showed in the screenshot.

This is by design. Hydra is an OAuth Server and delegates the login function to Kratos. Therefore, we need to use Hydra to access 127.0.0.1:4455/login.

So you will always have to access, 127.0.0.1:4455/login via Hydra in the hydra-consent branch.

You can use any other OAuth Client Library and then start the OAuth Authorization Flow by calling the 127.0.0.1:4455/auth/login endpoint which is the login endpoint for Hydra.

If you check out the main branch and access localhost:4455/login, you will notice that the login endpoint works because there is no Hydra.

Please note that for the main branch, it's localhost and not 127.0.0.1

Let me know if my explanation is clear or if you need any clarifications regarding this 🙇🏼‍♂️

[atreya2011]

I run docker-compose up --build - here I got error about (\"SECRETS_SYSTEM\" environment variable) is not set or changed then I did 'docker-compose down` then docker-compose up --build again. But unfortunately error was still present. As I understand two exec command is solving this problem

@DeamonMV Sorry, it took a while but I have managed to solve the above issue so that only ONE execution of docker-compose up is enough for the migrations to work! Also ensured that you get a Status Unauthorized error if you try to access the /login endpoint without starting an OAuth Authorization Flow.

Please ensure that you do the following:

git checkout hydra-consent && \
git pull -f origin hydra-consent

When you get the time, let me know if the issue has been resolved for you 🙇🏼‍♂️

[jeff0131]

This is excellent work 🥇 Thank you so much.

I'm am encountering an issue. In the main.go file on line 107 the returnTo is auth/login, which doesn't exist, so you get an error. When I changed it to /login it worked. I wonder if that is correct or if there is some reason that auth/login should work?

[DeamonMV]

Hello @atreya2011

I tested your changes. 1) I think, you forgot to add SECRETS_SYSTEM=youReallyNeedToChangeThis ENV to hydra container. Without this ENV container end up with an error, that ENV SECRETS_SYSTEM is missed

2) >Also ensured that you get a Status Unauthorized error if you try to access the /login endpoint without starting an OAuth Authorization Flow.

Yes, this seems working. But why it need?

3) I'm agree with @jeff0131, when I changed line 107

From: returnTo := "/auth/login?" + returnToParams.Encode() To: returnTo := "/login?" + returnToParams.Encode()

At the end of the login process, after I entered login and password and hit enter, I receiving Tokens and Expires In date/time.

---

I must say, that I'm not quit understand how it should work in production.

so far I have two question:

• how to automate this, to have it work without needing to run this command every time I need to login.

Then, execute the following to start the OAuth2 Authorize Code Flow

docker-compose exec hydra \
    hydra token user \
    --client-id auth-code-client \
    --client-secret secret \
    --endpoint http://127.0.0.1:4444/ \
    --port 5555 \
    --scope openid,offline

How it will be looks like in production. User will be needed every time to pass through some specific url to start Authorize Code Flow

• I'm I right that to store those tokens in browser, from hydra, I need some SPA webpage, written on react etc.

Because right now after tokens was achieved, I can't pass http://127.0.0.1/verified it says Unauthorized. And actually login process end on page with a Tokens.

I guess I'm missing something, but I can't understand what I need search to make all those process clear to me)

Anyway thank you for your effort 👍

[atreya2011]

This is excellent work 1st_place_medal Thank you so much.

I'm am encountering an issue. In the main.go file on line 107 the returnTo is auth/login, which doesn't exist, so you get an error. When I changed it to /login it worked. I wonder if that is correct or if there is some reason that auth/login should work?

@jeff0131 Thank you for pointing that out. That was a bug and I fixed it in my recent commit 🙇🏼‍♂️ git pull -f origin hydra-consent to apply the fix 🙏🏼

@DeamonMV

Thank you for your taking your time in confirming the recent updates 🙇🏼‍♂️

I think, you forgot to add SECRETS_SYSTEM=youReallyNeedToChangeThis ENV to hydra container. Without this ENV container end up with an error, that ENV SECRETS_SYSTEM is missed

I added this to the hydra-migrate service in the docker-compose.yml file. https://github.com/atreya2011/go-kratos-test/blob/73b85e76628448435189ce69d86bd2c4b11fbabf/docker-compose.yml#L30

The hydra service reads the config file which also contains SECRETS_SYSTEM value. Let me know if I have missed anything 🙏🏼 https://github.com/atreya2011/go-kratos-test/blob/hydra-consent/config/hydra.yml

I'm agree with @jeff0131, when I changed line 107

Yes that was a bug and I fixed it my recent commit. git pull -f origin hydra-consent to apply the fix 🙏🏼

At the end of the login process, after I entered login and password and hit enter, I receiving Tokens and Expires In date/time.

Great! Glad you got it working :)

how to automate this, to have it work without needing to run this command every time I need to login.

docker-compose exec hydra \
hydra token user \
--client-id auth-code-client \
--client-secret secret \
--endpoint http://127.0.0.1:4444/ \
--port 5555 \
--scope openid,offline

The above CLI command creates an example OAuth Client for you to just test whether kratos-hydra setup is working properly or not and you are correct that it cannot be used in production :) More information can be found here regarding the command: https://www.ory.sh/hydra/docs/next/cli/hydra-token-user/

How it will be looks like in production. User will be needed every time to pass through some specific url to start Authorize Code Flow I'm I right that to store those tokens in browser, from hydra, I need some SPA webpage, written on react etc.

I have actually made an example UI using Next.js + NextAuth. Next-Auth automatically starts the OAuth Authorization flow for the user when they click the Sign in button shown in the screenshot.

This is the repository: https://github.com/atreya2011/next-auth-kratos-hydra-test/tree/hydra-consent

To try out the example, Please ensure that you have the latest version of Node.js installed :)

1. First, create an OAuth Client in Hydra using the following command. Note that the callback URL is different now :)

   docker-compose exec hydra \
   hydra clients create \
   --endpoint http://127.0.0.1:4445 \
   --id next-auth-test-client \
   --secret secret \
   --grant-types authorization_code,refresh_token \
   --response-types code,id_token \
   --scope openid,offline \
   --callbacks http://127.0.0.1:3000/api/auth/callback/kratos-hydra

2. In a different terminal session, clone the repo (https://github.com/atreya2011/next-auth-kratos-hydra-test/tree/hydra-consent)
3. git checkout hydra-consent
4. cp .env.local.example .env.local
5. npm install && npm run dev
6. Open 127.0.0.1:3000 in your browser and you will see the following screen
7. Click on Sign in with Kratos & Hydra to start the OAuth Authorization Flow.

Let me know if you have any issues 🙇🏼‍♂️

[tsu84]

Hi, I tried to run the project with the node front end like with the instructions on the previous response, but what I get after clicking the button two time is a page http://127.0.0.1:4455/login?login_challenge=d85459f8d8f94908b40deabe2ceeef49 with Internal Server Error in the body.

What can be the problem?

[edit] I found out the error is thrown on line 247 in main.go: md, ok := clientRes.Payload.Metadata.(string)

[atreya2011]

@tsu84 Thank you for letting me know where the error was 🙏🏼 Just to double-check you are trying the hydra-consent branch and the error thrown is on this line right? https://github.com/atreya2011/go-kratos-test/blob/b0515620dc3e3eb9c9995b1bbd55f0cc2907c7ee/main.go#L247

If that is the case, can you try recreating the OAuth Client using the following curl command? The OAuth Client is probably missing the metadata which can only be set using the HTTP API provided by Hydra.

curl -X POST 'http://localhost:4445/clients' \
-H 'Content-Type: application/json' \
--data-raw '{
  "client_id": "auth-code-client",
  "client_name": "Auth Code Client",
  "grant_types": ["authorization_code","refresh_token"],
  "redirect_uris": ["http://127.0.0.1:5555/callback"],
  "response_types": ["code", "id_token"],
  "scope": "openid",
  "metadata": "{\"registration\":true,\"verification\":true}",
  "post_logout_redirect_uris": ["http://localhost:4455"]
}'

Let me know if it the login flow works 🙏🏼

[tsu84]

I used the snippet found on main.go, which is without metadata.verification (which throws a kratos error "No active session was found in this request" ) and post_logout_redirect_uris (which says the redirect uri is not already defined in default list): curl -X POST 'http://localhost:4445/clients' \ -H 'Content-Type: application/json' \ --data-raw '{ "client_id": "auth-code-client", "client_name": "Test OAuth2 Client", "client_secret": "secret", "grant_types": ["authorization_code", "refresh_token"], "redirect_uris": ["http://127.0.0.1:3000/api/auth/callback/kratos-hydra"], "response_types": ["code", "id_token"], "scope": "openid offline", "token_endpoint_auth_method": "client_secret_post", "metadata": "{\"registration\": true}" }'

I went further and now I'm stuck on CSRF mismatch on Kratos.

[atreya2011]

@tsu84 Hey sorry but give me some time to take a look at this 🙏🏼

[atreya2011]

@tsu84 Sorry for the delay. Can you try the following

1. Clone this repository (I guess you have already done this)
2. git checkout hydra-consent
3. docker system prune -a -f --volumes (to remove dangling containers and volumes etc.)
4. docker-compose up (wait for all containers to be up and running)
5. Open another terminal window
6. Create an OAuth Client using the following command

   curl -X POST 'http://localhost:4445/clients' \
   -H 'Content-Type: application/json' \
   --data-raw '{
   "client_id": "auth-code-client",
   "client_name": "Test OAuth2 Client",
   "client_secret": "secret",
   "grant_types": ["authorization_code", "refresh_token"],
   "redirect_uris": ["http://localhost:4455/dashboard"],
   "response_types": ["code", "id_token"],
   "scope": "openid offline",
   "token_endpoint_auth_method": "client_secret_post",
   "metadata": "{\"registration\": true}"
   }'

7. Open an incognito window using Chrome (or if there is such a mode in any other browser of your choice)
8. Open this URL http://localhost:4455/login
9. Register a new account by clicking the Registration link
10. Open this URL in a new tab http://localhost:4436 (Mailslurper) and click on the verification link
11. You can login with the newly created account

Let me know if you face any issues 🙏🏼 Please ensure that all URLs are localhost and NOT 127.0.0.1 as this may cause CSRF issues...

[karthnag-zee]

@tsu84 Sorry for the delay. Can you try the following

1. Clone this repository (I guess you have already done this)
2. git checkout hydra-consent
3. docker system prune -a -f --volumes (to remove dangling containers and volumes etc.)
4. docker-compose up (wait for all containers to be up and running)
5. Open another terminal window
6. Create an OAuth Client using the following command

curl -X POST 'http://localhost:4445/clients' \
-H 'Content-Type: application/json' \
--data-raw '{
  "client_id": "auth-code-client",
  "client_name": "Test OAuth2 Client",
  "client_secret": "secret",
  "grant_types": ["authorization_code", "refresh_token"],
  "redirect_uris": ["http://localhost:4455/dashboard"],
  "response_types": ["code", "id_token"],
  "scope": "openid offline",
  "token_endpoint_auth_method": "client_secret_post",
  "metadata": "{\"registration\": true}"
}'

7. Open an incognito window using Chrome (or if there is such a mode in any other browser of your choice)
8. Open this URL http://localhost:4455/login
9. Register a new account by clicking the Registration link
10. Open this URL in a new tab http://localhost:4436 (Mailslurper) and click on the verification link
11. You can login with the newly created account

Let me know if you face any issues 🙏🏼 Please ensure that all URLs are localhost and NOT 127.0.0.1 as this may cause CSRF issues...

Hello @atreya2011,

I have followed the steps you have mentioned above but i am getting HTTP 401 "Unauthorized OAuth Client" for the login challenge. Should I change anything else? Please let me know.

curl 'http://localhost:4455/login?login_challenge=854b0da0eebc4ba1b9259e52c62ef247' \
  -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
  -H 'Accept-Language: en-GB,en-US;q=0.9,en;q=0.8' \
  -H 'Connection: keep-alive' \
  -H 'Cookie: idp=MTY1ODMxNTc2MHxEdi1CQkFFQ180SUFBUkFCRUFBQVVmLUNBQUVHYzNSeWFXNW5EQTBBQzI5aGRYUm9NbE4wWVhSbEJuTjBjbWx1Wnd3dUFDeENkVzFoYldOQ2JuUndTREZKVG1oNlZucHRhWFF6TVdSMFREazRhMng2VWpoRGFYbGtRVXBxTXpRd1BRPT18doFiqbty5Cys3-oQmMxYfe6tzAnYwqj_OyZnfvGrBWo=; oauth2_authentication_csrf_insecure=MTY1ODMxNTc2MHxEdi1CQkFFQ180SUFBUkFCRUFBQVB2LUNBQUVHYzNSeWFXNW5EQVlBQkdOemNtWUdjM1J5YVc1bkRDSUFJR000T1RWbU5qQmlOVFEzTWpReVlUUTVOVFk0TXprM01EQTFNRGRtTXpoanzFaVK-E1MT2alAoVpy0yEFi0VcmyzuYpCDtMpyHSn6Hw==' \
  -H 'Sec-Fetch-Dest: document' \
  -H 'Sec-Fetch-Mode: navigate' \
  -H 'Sec-Fetch-Site: none' \
  -H 'Sec-Fetch-User: ?1' \
  -H 'Upgrade-Insecure-Requests: 1' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36' \
  -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="102", "Google Chrome";v="102"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"' \
  --compressed 

[atreya2011]

@karthnag-zee Sorry for the delay. Can you try following the instructions from the below link and let me know if it works 🙏🏼

https://github.com/atreya2011/go-kratos-test/tree/hydra-consent#readme

Please ensure that you checkout the hydra-consentbranch.

[jaredweinfurtner]

@karthnag-zee Sorry for the delay. Can you try following the instructions from the below link and let me know if it works 🙏🏼

https://github.com/atreya2011/go-kratos-test/tree/hydra-consent#readme

Please ensure that you checkout the hydra-consentbranch.

When I follow the README.md, I call:

curl -X POST 'http://localhost:4445/admin/clients' \
-H 'Content-Type: application/json' \
--data-raw '{
  "client_name": "Test OAuth2 Client",
  "client_secret": "secret",
  "grant_types": ["authorization_code", "refresh_token"],
  "redirect_uris": ["http://localhost:4455/dashboard"],
  "post_logout_redirect_uris": ["http://localhost:4455/login"],
  "response_types": ["code", "id_token"],
  "scope": "openid offline",
  "token_endpoint_auth_method": "client_secret_post",
}'          

I receive the response:

{"error":"error","error_description":"The error is unrecognizable"}

The reason is that the JSON is not valid due to a comma at the end of the last key/value pair:

"token_endpoint_auth_method": "client_secret_post", <----- here

[atreya2011]

@jaredweinfurtner Sorry for the late response. This PR should have fixed the problem you are facing! https://github.com/atreya2011/go-kratos-test/pull/9