FATAL: role "$POSTGRES_USER" does not exist

[dangernil]

Error message in nomad Postgres service stderr saying postgress_user does not exits

Docker env does show a postgres_user

ERROR MSG: FATAL: role "$POSTGRES_USER" does not exist

[zhenik]

@dangernil might need to investigate further the docker image

[claesgill]

Have been struggling with this issue. The error seems to disappear when using user provided username in the check stanza that is directly rendered into args (fixed in #53 ):

    service {
      ...
      check {
        ...
        command   = "/usr/local/bin/pg_isready"
        args      = ["-U", "${username}"]
        ...
      }

However for some reason it does not like using env variables in the check stanza (thus the issue):

    service {
      ...
      check {
        ...
        command   = "/usr/local/bin/pg_isready"
        args      = ["-U", "$POSTGRES_USER"]
        ...
      }

Problem with the latter is that I think we need to get the username from Vault, but I'm not sure how we can achieve this since the check stanza does not support using Consul template..

Suggestions would be dearly appreciated 🙏 @pdmthorsrud @zhenik @dangernil

[pdmthorsrud]

Considering the error msg FATAL: role "$POSTGRES_USER" does not exist, and that it seems to be looking for the string $POSTGRES user, are you sure it is all render properly/syntax is correct? I'm not entirely sure you can access env-variables like that in nomad

EDIT: Shouldn't it be "${POSTGRES_USER}"? https://www.nomadproject.io/docs/runtime/interpolation

[zhenik]

If I understood you correctly.

If you want access env variable from host container and avoid rendering issues with terraform, you can try to use double $$

args      = ["-U", "$$POSTGRES_USER"]

Also, it is important to notice the service stanza placement. It could be placed both under group and under task. It will influence check stanza at last. Documentation

I have checked that check is specified for concrete task here https://github.com/fredrikhgrelland/terraform-nomad-postgres/blob/master/conf/nomad/postgres.hcl#L38 , than the placement should not be a problem.

[pdmthorsrud]

If I understood you correctly.

If you want access env variable from host container and avoid rendering issues with terraform, you can try to use double $$

args      = ["-U", "$$POSTGRES_USER"]

Oooh, that might be it, I was thinking in the wrong context. Disregard my earlier comment, I am wrong!

[claesgill]

If I understood you correctly.

If you want access env variable from host container and avoid rendering issues with terraform, you can try to use double $$

args      = ["-U", "$$POSTGRES_USER"]

Also, it is important to notice the service stanza placement. It could be placed both under group and under task. It will influence check stanza at last. Documentation

I have checked that check is specified for concrete task here https://github.com/fredrikhgrelland/terraform-nomad-postgres/blob/master/conf/nomad/postgres.hcl#L38 , than the placement should not be a problem.

This resolves to the following error 😅 :

2020-11-13 13:49:10.371 UTC [114] FATAL:  role "$$POSTGRES_USER" does not exist

I will look more into the documentation you have linked 🤞

[pdmthorsrud]

Eyo, just wanted to mention that this is something that has actually surfaced again. To be frank, I'm guessing it never really went away. What implications it has I'm not sure of. Postgres seems to be working fine while doing queries directly to postgres at least

edit: if someone outside of Skatteetaten sees this I'm sorry, I just linked to an internal project in skatteetaten 🤷

[nadiia-caspar]

Will post it here if it can help someone. I had the same error in Kubernetes livenessProbe, that how I specified the command: command: ["/bin/bash", "-c", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB"]. Cheers!