dynatrace-oss / terraform-provider-dynatrace

Apache License 2.0
70 stars 33 forks source link

Export utility doesn't populate 'content' for dynatrace_document #581

Closed Rhamus98 closed 6 hours ago

Rhamus98 commented 2 days ago

Describe the bug When attempting to export the dynatrace_document resource, with the correct, additional, environment variables set:

- DT_CLIENT_ID
- DT_CLIENT_SECRET
- DT_ACCOUNT_ID

The resource is returned like this:

resource "dynatrace_document" "terraform-resource-name" {
  name      = "dashboard_name"
  type      = "dashboard"
  actor     = ""
  content   = ""
  owner     = "owner_id"
}

To Reproduce Steps to reproduce the behavior: We are executing the export as part of a GitHub Actions workflow step, whereby we retrieve the relevant secrets from hashicorp vault and pass them into a script:

jobs:
  dynatrace-sandbox-export:
    runs-on: self-hosted-rhel-runner
    steps:
      - name: Fetch Vault Secrets from ISD Observability
        id: vault_secret
        uses: hashicorp/vault-action@v2
        with:
          role: ${{ env.VAULT_ROLE }}
          url: ${{ env.VAULT_URL }}
          method: jwt
          namespace: ${{ env.VAULT_NAMESPACE }}
          tlsSkipVerify: true
          exportEnv: true
          secrets: |
            secrets_path/ sandbox | DYNATRACE_SB_API_TOKEN;
            secrets_path/ sandbox | DYNATRACE_SB_ENV_URL;
      - name: Fetch Vault Secrets from ISD Observability CTC Namespace
        id: vault_secret_oauth
        uses: hashicorp/vault-action@v2
        with:
          role:  ${{ env.VAULT_ROLE }}
          url: vault_url
          method: jwt
          namespace: ${{ env.VAULT_NAMESPACE }}
          tlsSkipVerify: true
          exportEnv: true
          secrets: |
            secrets_path/ oauth_client_id | DT_CLIENT_ID;
            secrets_path/ oauth_secret | DT_CLIENT_SECRET;
            secrets_path/ account_id | DT_ACCOUNT_ID;
      - name: Fetch Vault Secrets from Horizon
        id: vault_secret_2
        uses: hashicorp/vault-action@v2
        with:
          role:  ${{ env.VAULT_ROLE }}
          url: vault_url
          method: jwt
          namespace: ${{ env.VAULT_NAMESPACE }}
          tlsSkipVerify: true
          exportEnv: true
          secrets: |
            secret_path credentials | PAT;
      - name: Check out the repository to the runner
        uses: actions/checkout@v4  
      - name: Run a script to pull dashboards
        run: ./scripts/dt_backup.sh
        env:
          DYNATRACE_ENV_URL: "https://${{ env.DYNATRACE_SB_ENV_URL }}.live.dynatrace.com"
          DYNATRACE_API_TOKEN: ${{ env.DYNATRACE_SB_API_TOKEN }}
          DYNATRACE_TARGET_FOLDER: export/sandbox/dashboards
          GITHUB_TOKEN: ${{ env.PAT }}
          DASHBOARDS: true
          DT_CLIENT_ID: ${{ env.DT_CLIENT_ID }}
          DT_CLIENT_SECRET: ${{ env.DT_CLIENT_SECRET }}
          DT_ACCOUNT_ID: ${{ env.DT_ACCOUNT_ID }}

And the script that looks like this:

#!/bin/bash

git clone repo_url
cd horizon-dynatrace-backup

ENV=$(echo "$DYNATRACE_TARGET_FOLDER" | cut -d'/' -f2)
TYPE=$(echo "$DYNATRACE_TARGET_FOLDER" | cut -d'/' -f3)
TIMESTAMP=$(date +%Y-%m-%d-%H-%M-%S)
dashboard_location="./export/$ENV/dashboards/"

echo "DYNATRACE_TARGET_FOLDER is : $DYNATRACE_TARGET_FOLDER"

git checkout -b dynatrace_terraform_backup_${TIMESTAMP}_${ENV}_${TYPE}
rm -rf export && mkdir -p $DYNATRACE_TARGET_FOLDER

if [ "$DASHBOARDS" == true ];then
    ../scripts/terraform-provider-dynatrace_v1.69.1 -export -import-state-v2 dynatrace_json_dashboard dynatrace_document -exclude dynatrace_dashboard_sharing
    ../scripts/delete_tokens.sh $dashboard_location
else
    if [ $ENV == "dev" ];then
        ../scripts/terraform-provider-dynatrace_v1.69.1 -export -import-state-v2 -exclude dynatrace_credentials dynatrace_oneagent_features
        ../scripts/terraform-provider-dynatrace_v1.69.1 -export -import-state-v2 dynatrace_automation_workflow
    else
        ../scripts/terraform-provider-dynatrace_v1.69.1 -export -import-state-v2 -exclude dynatrace_credentials
        ../scripts/terraform-provider-dynatrace_v1.69.1 -export -import-state-v2 dynatrace_automation_workflow
    fi
fi

Expected behavior I'd expect, amongst other thing, for the dynatrace_document resource to be exported per the document, and for the contents block to be a JSON body for example, instead of blank: image

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

Reinhard-Pilz-Dynatrace commented 1 day ago

Hello @Rhamus98

We will need to capture the HTTP traffic that happens when you're running the export. To capture that traffic you can set the environment variables

DYNATRACE_DEBUG=true
DYNATRACE_LOG_HTTP=terraform-provider-dynatrace.http.log
DYNATRACE_HTTP_RESPONSE=true

and then execute

terraform-provider-dynatrace_v1.##.# -export dynatrace_document

That will produce a file terraform-provider-dynatrace.http.log (and likely a file terraform-provider-dynatrace.http.err.log).

But I want you to wait with that until we have pushed out the release scheduled for tomorrow. It contains a couple of improvements about HTTP traffic logging I would like to be active for that.

Because this is GitHub repo is visible for everyone, I'd suggest to either send the captured logs to terraform@dynatrace.com or to open a Dynatrace Support Ticket for exchanging such information. These logs will contain details about your Dynatrace configuration, and you don't want that to be available out in the public.

My current guess is that your issue is related to permissions on the dashboards in question. But the logs should tell us a bit more.

best regards, Reinhard

jparrottOneStream commented 18 hours ago

I am also having this exact problem with the export utility. I am using 1.69.1. It exports the information block about the resource but leaves the content field blank.

# ID 12345678-ba5e-462b-b2aa-7d0b6d3af61e
resource "dynatrace_document" "test_dashboard" {
  name    = "test_dashboard"
  type    = "dashboard"
  actor   = ""
  content = ""
  owner   = "12345678-0a9c-4a69-a17b-7655102870a5"
  private = true
}

I tested this out in 1.69.0 and it was still broken. It worked when I tried an earlier version (1.65.0).

Reinhard-Pilz-Dynatrace commented 6 hours ago

Hi @Rhamus98 and @jparrottOneStream

I have good news here. The upcoming release of the provider (planned for today) will fix this issue.

Thanks for your comment, @jparrottOneStream. It made me revisit past and upcoming changes to the relevant REST client. Before that I was pretty puzzled about why I wasn't able to reproduce the issue.

Just drop us a message in case today's release doesn't work out as expected and I'll reopen the ticket.

Rhamus98 commented 6 hours ago

Thank you @Reinhard-Pilz-Dynatrace and @jparrottOneStream. I’ll test this and post an update here!

jparrottOneStream commented 23 minutes ago

Awesome! Thanks for the quick fix. I'm glad my insight was able to help. I will try it out and post an update here.