search

timohirt / terraform-provider-hetznerdns

Terraform provider for Hetzner DNS
Mozilla Public License 2.0
109 stars 21 forks source link

When TXT record contains string ; API will create and return escaped string with \" #7

Closed insekticid closed 4 years ago

insekticid commented 4 years ago

image

resource "hetznerdns_record" "txtdkim" {
    zone_id = hetznerdns_zone.zone1.id
    name    = "google._domainkey"
    value   = "anything;with;param"
    type    = "TXT"
    ttl     = 3600
}

terraform plan will produce neverending update

  # module.e_records.hetznerdns_record.txtdkim[0] will be updated in-place
  ~ resource "hetznerdns_record" "txtdkim" {
        id      = "xxx"
        name    = "google._domainkey"
        ttl     = 3600
        type    = "TXT"
      ~ value   = "\"v=DKIM1;k=rsa;p=OLOL\"" -> "v=DKIM1;k=rsa;p=OLOL"
        zone_id = "xxx"
    }

btw thank you for your work👌 , I wrote article here https://www.exploit.cz/hetzner-dns-via-terraform/

klausenbusk commented 4 years ago

I just sent a mail to Hetzner support, as it looks like a API bug.

klausenbusk commented 4 years ago

Hetzner support:

thank you for the bug report, we notified our developers about the issue and they are about to investigate.
timohirt commented 4 years ago

Hey @klausenbusk! Did the Hetzner support folks respond?

klausenbusk commented 4 years ago

Hey @klausenbusk! Did the Hetzner support folks respond?

I haven't heard anything since the initial response.

timohirt commented 4 years ago

Hey @insekticid! I tried to reproduce today with the following terraform:

data "hetznerdns_zone" "zone1" {
    name = "my-domain.de"
}

resource "hetznerdns_record" "txtdkim" {
    zone_id = data.hetznerdns_zone.zone1.id
    name    = "google._domainkey"
    value   = "anything;with;param"
    type    = "TXT"
    ttl     = 3600
}

It worked well. Not sure if Hetzner already fixed the bug @klausenbusk reported. Does it work for you as well or do you still see errors?

timohirt commented 4 years ago

Updating the resource works as well.

$ terraform apply

Terraform will perform the following actions:

  # hetznerdns_record.txtdkim will be updated in-place
  ~ resource "hetznerdns_record" "txtdkim" {
        id      = "3d60921a49eb384b6335766a48d418ee"
        name    = "google._domainkey"
        ttl     = 3600
        type    = "TXT"
      ~ value   = "\"anything;with;param\"" -> "\"v=DKIM1;k=rsa;p=OLOL\""
        zone_id = "rMu2waTJPbHr45psPMZxHV"
    }

Plan: 0 to add, 1 to change, 0 to destroy.

hetznerdns_record.txtdkim: Modifying... [id=3d60921a49eb384b6335766a48d418ee]
hetznerdns_record.txtdkim: Modifications complete after 1s [id=3d60921a49eb384b6335766a48d418ee]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

$ terraform apply                                          

data.hetznerdns_zone.zone1: Refreshing state... [id=rMu2waTJPbHr45psPMZxHV]
hetznerdns_record.txtdkim: Refreshing state... [id=3d60921a49eb384b6335766a48d418ee]

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
klausenbusk commented 4 years ago

It worked well. Not sure if Hetzner already fixed the bug @klausenbusk reported. Does it work for you as well or do you still see errors?

I can reproduce the issue with:

provider "hetznerdns" {
  apitoken = "xxxx"
}

resource "hetznerdns_record" "txtdkim" {
    zone_id = "LFwsMkxWxQeCfBejGdHUWd"
    name    = "google._domainkey"
    value   = "anything;with;param"
    type    = "TXT"
    ttl     = 3600
}

Are you using terraform 0.12.x?

insekticid commented 4 years ago

Issue is still here 

 resource "hetznerdns_record" "txtspf" {
        id      = "4f3a6fdea95e50f2788ccdc9e04a5fd1"
        name    = "@"
        ttl     = 3600
        type    = "TXT"
      ~ value   = "\"v=spf1 include:_spf.google.com ~all\"" -> "v=spf1 include:_spf.google.com ~all"
        zone_id = "2ZB3bTCTMFR9WckY6Y7fEQ"
    }

Terraform v0.12.29 image

You can see, that quotes are in Hetzner DNS too, but only in problematic strings image

Edit: Migrated to Terraform 0.13 and the same issue here + my new article, how to migrate https://www.exploit.cz/how-to-migrate-from-plugins-to-terraform-0-13-registry-providers/

and0x000 commented 4 years ago

I think the escaped quotation mark is correct in terms of DNS and JSON. Some characters in a TXT record need to be escaped, as a TXT record may be a valid DNS information, but not necessarily a valid information for the application, that abuses TXT as a transport (such as SPF).

Example:

@    IN    TXT    v=spf1 include:_spf.google.com ~all

This would be a valid TXT record, but it would drop all the whitespaces, thus rendering the string incorrect in terms of SPF.

Putting the characters inside quotation marks is one way of escaping and preserves the whitespaces for the SPF context. Therefore, the quotation marks are part of the TXT record, which is why having them in the JSON would be legit from my point of view. In JSON, the quotation marks are again part of the data and need to be escaped for obvious reasons.

References:

timohirt commented 4 years ago

I tried to reproduce again and failed. I think I need your help, folks. 

resource "hetznerdns_record" "txtdkim" {
    zone_id = data.hetznerdns_zone.zone1.id
    name    = "google._domainkey"
    value   = "anything:with:param"
    type    = "TXT"
    ttl     = 3600
}

I created the record above with Terraform and then added quotes in DNS console.

image

Now, terraform plan shows the following output:

Terraform will perform the following actions:

  # hetznerdns_record.txtdkim will be updated in-place
  ~ resource "hetznerdns_record" "txtdkim" {
        id      = "3d60921a49eb384b6335766a48d418ee"
        name    = "google._domainkey"
        ttl     = 3600
        type    = "TXT"
      ~ value   = "\"anything:with:param\"" -> "anything:with:param"
        zone_id = "rMu2waTJPbHr45psPMZxHV"
    }

Plan: 0 to add, 1 to change, 0 to destroy.

This look like what @insekticid reported. Now I added quotes in the Terraform resource:

resource "hetznerdns_record" "txtdkim" {
    zone_id = data.hetznerdns_zone.zone1.id
    name    = "google._domainkey"
    value   = "\"anything:with:param\""
    type    = "TXT"
    ttl     = 3600
}

And the terraform plan finds no changes:

No changes. Infrastructure is up-to-date.

Do you use quotes in your resources? And if yes, could you please provide a complete example?

timohirt commented 4 years ago

Migrated to Terraform 0.13 and the same issue here + my new article, how to migrate https://www.exploit.cz/how-to-migrate-from-plugins-to-terraform-0-13-registry-providers/

@insekticid would you mind if I'm putting a link to your migration guide in README?

insekticid commented 4 years ago

Lets do this!

On Mon, 24 Aug 2020 at 21:45, Timo Hirt notifications@github.com wrote:

Migrated to Terraform 0.13 and the same issue here + my new article, how to migrate

https://www.exploit.cz/how-to-migrate-from-plugins-to-terraform-0-13-registry-providers/

@insekticid https://github.com/insekticid would you mind if I'm putting a link to your migration guide in README?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/timohirt/terraform-provider-hetznerdns/issues/7#issuecomment-679328944, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABLJPEEMNTWC5QA47ABV6TSCK7O3ANCNFSM4PFRM5XQ .

klausenbusk commented 4 years ago

So I got a response from Hetzner: we decided NOT to fix this, as we do not consider this a bug on our end. with a explanation similar to https://github.com/timohirt/terraform-provider-hetznerdns/issues/7#issuecomment-676446652 .. So it is a feature ;)

timohirt commented 4 years ago

Good to know. Thanks for asking them.

I think it works fine with escaped quotes in Terraform resources, as described in https://github.com/timohirt/terraform-provider-hetznerdns/issues/7#issuecomment-679328404.. Or I'm just unable to reproduce.

klausenbusk commented 4 years ago

Full response from Hetzner (I didn't want to share it without asking):

after some internal discussion, we decided NOT to fix this, as we do not consider this a bug on our end.

TXT records are allowed to contain quotation marks. In case of the use of a TXT record for SPF they may be even required to escape e.g. whitespaces.

We are aware that it may not look that nice, if you have another set of escaped quotation marks inside a JSON string, which itself is contained within quotation marks.

However, we decided to rather go for the real representation rather than depending it on an error-prone guess which of the various use cases of a TXT record is intended by you as a user.

We will make this a bit more pointed in the API documentation, though.

svenstaro commented 4 years ago

So perhaps this particular issue shall be fixed then via docs?

timohirt commented 4 years ago

@svenstaro to be honest, I'm not sure if this is a bug. If I want the value of a TXT record quoted, the value needs to be quoted in Terraform as well. So, "v=DKIM1;k=rsa;p=OLOL" at Hetzner DNS would be "\"v=DKIM1;k=rsa;p=OLOL\"" in Terraform. If I do this, the DNS record is created as expected and Terraform finds no changes if I run it again.

This behavior is what I expect in this case. What would you expect?

svenstaro commented 4 years ago

Actually that's fair. I just thought since apparently someone thought to make this bug to discuss this issue, perhaps a mention or example for proper quoting might be nice to give people some direction?

timohirt commented 4 years ago

I documented how to manage TXT records with values in quotes: https://github.com/timohirt/terraform-provider-hetznerdns/commit/1d860ebf975d27c6353f8690d4f0612cd483f0ee

insekticid commented 4 years ago

Issue is still here in v 1.0.7

locals {
dkim    = "\"v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAhTo5UKCr5cHhmne2ZZX5swOlNP794OWVxmhPt0hdsMaYQ9d1uJ86u4qdwuu6M7DCV/pRg4+XdSAeSGOm0O6k+oBSrkzk+ijGuCjDSKNrCsDEJwyVBkbnat2mDtKednRWolIZmUP1Rs9pNOKcq8FJI52Ewc5imeX1T0agawtRB6Q0RoJOoEo4Y0fNeDOFTcn9YE7qgl2dKjo7ng8Ty/na8IBMqIjbzVaKpKQnIKfCwopO4cF7fzDHuqxSCxD3CAidgUbdHarwgC5M7d43GnWmGwgA5PhnE0iUNASbbfStY4PSHXeq3+FO2euC348/Vk6WOE9tFTEFiF/oMq5qgm5yEwIDAQAB\""
}

resource "hetznerdns_record" "txtdkim" {
    zone_id = hetznerdns_zone.zone1.id
    name    = "google._domainkey"
    value   = local.dkim
    type    = "TXT"
    ttl     = 3600

    count   = local.dkim != "" ? 1 : 0
}

result:


  # hetznerdns_record.txtdkim[0] will be updated in-place
 ~ resource "hetznerdns_record" "txtdkim" {
        id      = "c6868f26cecf0bc125ebb82aefb34be2"
        name    = "google._domainkey"
        ttl     = 3600
        type    = "TXT"
      ~ value   = "\"v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAhTo5UKCr5cHhmne2ZZX5swOlNP794OWVxmhPt0hdsMaYQ9d1uJ86u4qdwuu6M7DCV/pRg4+XdSAeSGOm0O6k+oBSrkzk+ijGuCjDSKNrCsDEJwyVBkbnat2mDtKednRWolIZmUP1Rs9pNOKcq8FJI52Ewc5imeX1T0agawtRB6Q0RoJOoEo4Y0fNeDOFTcn9Y\" \"E7qgl2dKjo7ng8Ty/na8IBMqIjbzVaKpKQnIKfCwopO4cF7fzDHuqxSCxD3CAidgUbdHarwgC5M7d43GnWmGwgA5PhnE0iUNASbbfStY4PSHXeq3+FO2euC348/Vk6WOE9tFTEFiF/oMq5qgm5yEwIDAQAB\" " -> "\"v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAhTo5UKCr5cHhmne2ZZX5swOlNP794OWVxmhPt0hdsMaYQ9d1uJ86u4qdwuu6M7DCV/pRg4+XdSAeSGOm0O6k+oBSrkzk+ijGuCjDSKNrCsDEJwyVBkbnat2mDtKednRWolIZmUP1Rs9pNOKcq8FJI52Ewc5imeX1T0agawtRB6Q0RoJOoEo4Y0fNeDOFTcn9YE7qgl2dKjo7ng8Ty/na8IBMqIjbzVaKpKQnIKfCwopO4cF7fzDHuqxSCxD3CAidgUbdHarwgC5M7d43GnWmGwgA5PhnE0iUNASbbfStY4PSHXeq3+FO2euC348/Vk6WOE9tFTEFiF/oMq5qgm5yEwIDAQAB\""
        zone_id = "hbVXC5cdWQFTwe5n8aWs38"
    }

Plugin version 1.1.0 ends up with error - hashicorp/terraform:0.13.1 Error: rpc error: code = Unavailable desc = transport is closing

2020-08-31T05:35:39.891Z [DEBUG] plugin: starting plugin: path=.terraform/plugins/registry.terraform.io/timohirt/hetznerdns/1.1.0/linux_amd64/terraform-provider-hetznerdns_v1.1.0 args=[.terraform/plugins/registry.terraform.io/timohirt/hetznerdns/1.1.0/linux_amd64/terraform-provider-hetznerdns_v1.1.0]
2020-08-31T05:35:39.894Z [DEBUG] plugin: plugin started: path=.terraform/plugins/registry.terraform.io/timohirt/hetznerdns/1.1.0/linux_amd64/terraform-provider-hetznerdns_v1.1.0 pid=35
2020-08-31T05:35:39.894Z [DEBUG] plugin: waiting for RPC address: path=.terraform/plugins/registry.terraform.io/timohirt/hetznerdns/1.1.0/linux_amd64/terraform-provider-hetznerdns_v1.1.0
2020-08-31T05:35:39.906Z [INFO]  plugin.terraform-provider-hetznerdns_v1.1.0: configuring server automatic mTLS: timestamp=2020-08-31T05:35:39.905Z
2020-08-31T05:35:39.943Z [DEBUG] plugin: using plugin: version=5
2020-08-31T05:35:39.943Z [DEBUG] plugin.terraform-provider-hetznerdns_v1.1.0: plugin address: network=unix address=/tmp/plugin097001290 timestamp=2020-08-31T05:35:39.943Z
2020/08/31 05:35:40 [TRACE] BuiltinEvalContext: Initialized "provider[\"registry.terraform.io/timohirt/hetznerdns\"]" provider for provider["registry.terraform.io/timohirt/hetznerdns"]
2020/08/31 05:35:40 [TRACE] eval: *terraform.EvalOpFilter
2020/08/31 05:35:40 [TRACE] eval: *terraform.EvalSequence
2020/08/31 05:35:40 [TRACE] eval: *terraform.EvalGetProvider
2020/08/31 05:35:40 [TRACE] eval: *terraform.EvalValidateProvider
2020/08/31 05:35:40 [TRACE] buildProviderConfig for provider["registry.terraform.io/timohirt/hetznerdns"]: using explicit config only
2020/08/31 05:35:40 [TRACE] GRPCProvider: GetSchema
2020-08-31T05:35:40.015Z [TRACE] plugin.stdio: waiting for stdio data
2020-08-31T05:35:40.016Z [WARN]  plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unimplemented desc = unknown service plugin.GRPCStdio"
timohirt commented 4 years ago

Have you noticed there is a space in the value?

8/Vk6WOE9tFTEFiF/oMq5qgm5yEwIDAQAB\" " -> "\"v=DKIM1; k=rsa...

If you compare the last chars of the values it becomes more clear.

So there is a difference between the value at Hetzer and the Terraform file. That's why Terraform wants to update the record, right?

insekticid commented 4 years ago

@timohirt juste tell me how to use this string:

v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAhTo5UKCr5cHhmne2ZZX5swOlNP794OWVxmhPt0hdsMaYQ9d1uJ86u4qdwuu6M7DCV/pRg4+XdSAeSGOm0O6k+oBSrkzk+ijGuCjDSKNrCsDEJwyVBkbnat2mDtKednRWolIZmUP1Rs9pNOKcq8FJI52Ewc5imeX1T0agawtRB6Q0RoJOoEo4Y0fNeDOFTcn9Y\" \"E7qgl2dKjo7ng8Ty/na8IBMqIjbzVaKpKQnIKfCwopO4cF7fzDHuqxSCxD3CAidgUbdHarwgC5M7d43GnWmGwgA5PhnE0iUNASbbfStY4PSHXeq3+FO2euC348/Vk6WOE9tFTEFiF/oMq5qgm5yEwIDAQAB\" " -> "\"v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAhTo5UKCr5cHhmne2ZZX5swOlNP794OWVxmhPt0hdsMaYQ9d1uJ86u4qdwuu6M7DCV/pRg4+XdSAeSGOm0O6k+oBSrkzk+ijGuCjDSKNrCsDEJwyVBkbnat2mDtKednRWolIZmUP1Rs9pNOKcq8FJI52Ewc5imeX1T0agawtRB6Q0RoJOoEo4Y0fNeDOFTcn9YE7qgl2dKjo7ng8Ty/na8IBMqIjbzVaKpKQnIKfCwopO4cF7fzDHuqxSCxD3CAidgUbdHarwgC5M7d43GnWmGwgA5PhnE0iUNASbbfStY4PSHXeq3+FO2euC348/Vk6WOE9tFTEFiF/oMq5qgm5yEwIDAQAB

there is no way how to insert it correctly with this terraform plugin.

timohirt commented 4 years ago

@insekticid there are quotes in the first string (value of p):

cn9Y\" \"E7qg

To me this looks invalid. I'm not a DKIM expert tbh. This doc says that the value of p is a base64 encoded string. Base64 doesn't use spaces or quotes. Could you double check if the value is correct?

insekticid commented 4 years ago

@timohirt sorry, my mistake, correct record should be 

v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAhTo5UKCr5cHhmne2ZZX5swOlNP794OWVxmhPt0hdsMaYQ9d1uJ86u4qdwuu6M7DCV/pRg4+XdSAeSGOm0O6k+oBSrkzk+ijGuCjDSKNrCsDEJwyVBkbnat2mDtKednRWolIZmUP1Rs9pNOKcq8FJI52Ewc5imeX1T0agawtRB6Q0RoJOoEo4Y0fNeDOFTcn9YE7qgl2dKjo7ng8Ty/na8IBMqIjbzVaKpKQnIKfCwopO4cF7fzDHuqxSCxD3CAidgUbdHarwgC5M7d43GnWmGwgA5PhnE0iUNASbbfStY4PSHXeq3+FO2euC348/Vk6WOE9tFTEFiF/oMq5qgm5yEwIDAQAB
timohirt commented 4 years ago

Thanks @insekticid! I finally understood what's the problem. So, first. I think that quotes in Terraform are required if the DNS record should be quoted at Hetzner. Comment https://github.com/timohirt/terraform-provider-hetznerdns/issues/7#issuecomment-676446652 explains why this is and I agree.

However, there is another issue. I created a DNS record via API (bypassing Terraform) and used a TXT value with the same length as yours.

image

As you can see, there are two quotes in the middle of the string and I didn't put them there. This is something Hetzner has to look into. I'll create a new issue #13