search

qdm12 / ddns-updater

Container to update DNS records periodically with WebUI for many DNS providers
https://hub.docker.com/r/qmcgaw/ddns-updater/
MIT License
1.46k stars 146 forks source link

GCP request is missing required authentication credential #704

Open linchenqian opened 2 months ago

linchenqian commented 2 months ago

TLDR: using GCP provider is unable to update due to 401 error "CREDENTIALS_MISSING"

  1. Is this urgent: Yes

  2. DNS provider(s) you use: GCP

  3. Program version:

    Running version 2.6.0 built on 2024-02-03T18:57:00Z (commit c338c28ce3ab3aa9948f0fe133307b75fe6d34d2)

  4. What are you using to run the container: docker in Mac / windows binary(exe file) / container in TrueNas

  5. Extra information (optional)

Logs:

2024-04-30T00:56:09-04:00 INFO ipv4 address of <mydomain.com> is 108.xxx.xxx.xxx and your ipv4 address  is 108.xx.xxx.xx
2024-04-30T00:56:09-04:00 INFO Updating record [domain: <mydomain.com> | host: @ | provider: gcp | ip: ipv4] to use 108.xx.xxx.xx
2024-04-30T00:56:09-04:00 ERROR getting record resource set: googleapi: Error 401: Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
Details:
[
  {
    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
    "domain": "googleapis.com",
    "metadata": {
      "method": "cloud.dns.api.v1.ResourceRecordSetsService.Get",
      "service": "dns.googleapis.com"
    },
    "reason": "CREDENTIALS_MISSING"
  }
]

More details:
Reason: required, Message: Login Required.

Configuration file:

{
  "settings": [
    {
      "provider": "gcp",
      "project": "my-project",
      "zone": "my-zone",
      "credentials": {
        "type": "service_account",
        "project_id": "myproject",
        "private_key_id": "abcd123456",
        "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvgI..........MqGX/\n-----END PRIVATE KEY-----\n",
        "client_email": "ddns-updater-xxxxxx@xxxxxxx.iam.gserviceaccount.com",
        "client_id": "10163123456789",
        "auth_uri": "https://accounts.google.com/o/oauth2/auth",
        "token_uri": "https://oauth2.googleapis.com/token",
        "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
        "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/......",
        "universe_domain": "googleapis.com"
      },
      "domain": "mydomain.com",
      "host": "@",
      "ip_version": "ipv4",
      "ipv6_suffix": ""
    }
  ]
}

Host OS: Windows10/MacOS/Debian12

qdm12 commented 2 months ago

Update your image, Running version latest built on 2020-03-13T01:30:06Z is 4 years old.

linchenqian commented 2 months ago

Sorry forgot to update that line in the template, it's actually 2.6.0 built on 2024-02-03

Running version 2.6.0 built on 2024-02-03T18:57:00Z (commit c338c28ce3ab3aa9948f0fe133307b75fe6d34d2)
linchenqian commented 2 months ago

can you check again please?

qdm12 commented 2 months ago

Can you try with the latest image (pull it)? I recently upgraded the gcp dependency, maybe this solves it?

linchenqian commented 2 months ago

updated to the latest version, still having same issue:

Full logs:

========================================
========================================
============= ddns-updater =============
========================================
=========== Made with ❤️ by ============
======= https://github.com/qdm12 =======
========================================
========================================

Running version latest built on 2024-05-03T07:56:04.164Z (commit b131c3d)

🔧 Need help? https://github.com/qdm12/ddns-updater/discussions/new
🐛 Bug? https://github.com/qdm12/ddns-updater/issues/new
✨ New feature? https://github.com/qdm12/ddns-updater/issues/new
☕ Discussion? https://github.com/qdm12/ddns-updater/discussions/new
💻 Email? quentin.mcgaw@gmail.com
💰 Help me? https://www.paypal.me/qmcgaw https://github.com/sponsors/qdm12
2024-05-03T09:16:19Z INFO Settings summary:
├── HTTP client
|   └── Timeout: 10s
├── Update
|   ├── Period: 5m0s
|   └── Cooldown: 5m0s
├── Public IP fetching
|   ├── HTTP enabled: yes
|   ├── HTTP IP providers
|   |   └── all
|   ├── HTTP IPv4 providers
|   |   └── all
|   ├── HTTP IPv6 providers
|   |   └── all
|   ├── DNS enabled: yes
|   ├── DNS timeout: 3s
|   └── DNS over TLS providers
|       └── all
├── Resolver: use Go default resolver
├── Server
|   ├── Listening address: :8000
|   └── Root URL: /
├── Health
|   └── Server listening address: 127.0.0.1:9999
├── Paths
|   └── Data directory: /updater/data
├── Backup: disabled
└── Logger
    ├── Level: info
    └── Caller: hidden
2024-05-03T09:16:19Z INFO reading JSON config from file /updater/data/config.json
2024-05-03T09:16:19Z INFO Found single setting to update record
2024-05-03T09:16:19Z INFO Reading history from database: domain xxxx.xxx host @ ipv4
2024-05-03T09:16:19Z INFO [backup] disabled
2024-05-03T09:16:19Z INFO [healthcheck server] listening on 127.0.0.1:9999
2024-05-03T09:16:19Z INFO [http server] listening on :8000
2024-05-03T09:16:20Z INFO obtaining ipv4 address succeeded after 1 failed try
2024-05-03T09:16:20Z INFO ipv4 address of xxxx.xxx is 108.123.111.111 and your ipv4 address  is 108.123.222.233
2024-05-03T09:16:20Z INFO Updating record [domain: xxxx.xxx | host: @ | provider: gcp | ip: ipv4] to use 108.123.222.233
2024-05-03T09:16:20Z ERROR getting record resource set: googleapi: Error 401: Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
Details:
[
  {
    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
    "domain": "googleapis.com",
    "metadata": {
      "method": "cloud.dns.api.v1.ResourceRecordSetsService.Get",
      "service": "dns.googleapis.com"
    },
    "reason": "CREDENTIALS_MISSING"
  }
]

More details:
Reason: required, Message: Login Required.
qdm12 commented 2 months ago

Last questions before I jump in verifying the code, was it ever working before for you? Have you tried with another program to verify your credentials are correct? Thanks!

linchenqian commented 2 months ago

was it ever working before for you?

No, this is first time I use ddns-updater. I tried on multiple platform, all result in same error.

Have you tried with another program to verify your credentials are correct?

Yes, I have verified the credentials with gcloud cli

root@14d144a2678b:/home# cat creds.json 
{
                "type": "service_account",
                "project_id": ......
            }
root@14d144a2678b:/home# gcloud auth activate-service-account --key-file ./creds.json 
Activated service account credentials for: [ddns-updater-xxxxxxx@xxxx-xxx.iam.gserviceaccount.com]
root@14d144a2678b:/home# gcloud auth print-access-token
ya29.c.c0AY_VpZgwgNS......
root@14d144a2678b:/home# gcloud dns record-sets describe xxxx.xxx. --zone=my-zone --type=A --project=my-project
NAME        TYPE  TTL  DATA
xxxx.xxx.  A     300  100.0: 108.xx.xxx.xx

and also I tried copy part of the source code from update.go to a new program for testing, and it was able to pass the point where the error was thrown (getResourceRecordSet). and get result from google cloud dns. here is the code of the test program:

package main

import (
    "encoding/json"
    "errors"
    "fmt"
    "net/http"
    "github.com/qdm12/ddns-updater/internal/provider/constants"
    ddnserrors "github.com/qdm12/ddns-updater/internal/provider/errors"
    "github.com/qdm12/goshutdown"
    clouddns "google.golang.org/api/dns/v1"
    "google.golang.org/api/googleapi"
    "google.golang.org/api/option"
)

func main() {
    // Raw JSON string
    raw_creds := `{
        "type": "service_account",
        "project_id": ......
    }`

    // Assign the raw JSON string to a variable of type json.RawMessage
    credentials := json.RawMessage(raw_creds)

    _, ctx, _ := goshutdown.NewGoRoutineHandler("runner")
    ddnsService, err := clouddns.NewService(ctx, option.WithCredentialsJSON(credentials))
    if err != nil {
        fmt.Printf("Error creating DNS service: %s\n", err)
        return
    }

    rrSetsService := clouddns.NewResourceRecordSetsService(ddnsService)

    fqdn := fmt.Sprintf("%s.%s.", "", "xxx.xxx")
    recordType := constants.A
    recordResourceSet, err := getResourceRecordSet(rrSetsService, fqdn, recordType)
    if err != nil {
        fmt.Printf("Error getting resource record set: %s\n", err)
        return
    }

    for _, rrdata := range recordResourceSet.Rrdatas {
        if rrdata == "108.222.111.002" {
            // already up to date
            fmt.Print("ip ")
        }
    }
}

func getResourceRecordSet(rrSetsService *clouddns.ResourceRecordSetsService,
    fqdn, recordType string) (resourceRecordSet *clouddns.ResourceRecordSet, err error) {
    resourceRecordSet, err = rrSetsService.Get("my-project", "my-zone", fqdn, recordType).Do()
    if err != nil {
        googleAPIError := new(googleapi.Error)
        if errors.As(err, &googleAPIError) && googleAPIError.Code == http.StatusNotFound {
            return nil, fmt.Errorf("%w: %w", ddnserrors.ErrRecordResourceSetNotFound, err)
        }
        return nil, err
    }

    return resourceRecordSet, nil
}
qdm12 commented 1 month ago

Hmm strange, your code looks rather similar to the one in ddns-updater, maybe it's just the json credentials not being parsed/read correctly. I've done two things:

Thanks!

qdm12 commented 2 weeks ago

If you modify the line

ddnsService, err := clouddns.NewService(ctx, option.WithCredentialsJSON(credentials))

to

ddnsService, err := clouddns.NewService(ctx, option.WithCredentialsJSON(credentials), option.WithHTTPClient(http.DefaultClient))

Does it still work?

I'm digging in the google sdk to hopefully throw it out, and it seems like plugging in your own client just does not care about authentication; so that could be why it doesn't work??

qdm12 commented 2 weeks ago

Hi @linchenqian I just pushed a84a1d664a02b09b9313cb13e1bf8a1acbbb73b1 which removes the dependency on the SDK, and it should work (with the custom http client, oauth2, the credentials.json, everything, hopefully!). Please try it with docker and the latest image (docker pull qmcgaw/ddns-updater). Thanks!