GoogleCloudPlatform / terraformer

CLI tool to generate terraform files from existing infrastructure (reverse Terraform). Infrastructure to Code
Apache License 2.0
12.58k stars 1.64k forks source link

Unable to import DataDog dashboard #1375

Closed stiliajohny closed 1 year ago

stiliajohny commented 2 years ago

hi all,

while I am able to import the dashboard from DataDog, there is one that I cant,

Here is the error I am getting.

Thank you in advance

Starting import of 'xxx-xxx-xxx'...
2022/07/11 13:43:40 datadog importing... dashboard
2022/07/11 13:43:41 datadog done importing dashboard
2022/07/11 13:43:41 Number of resources for service dashboard: 1
2022/07/11 13:43:41 Refreshing state... datadog_dashboard.tfer--dashboard_gct-n3v-vca
2022/07/11 13:43:41 object contains unparsed element: map[compute:map[aggregation:count] data_source:ci_tests group_by:[map[facet:error.type limit:100 sort:map[aggregation:count order:desc]]] indexes:[*] name:query1 search:map[query:@test.status:fail @ci.pipeline.name:"Test on PR"]]
2022/07/11 13:43:41 WARN: Fail read resource from provider, wait 300ms before retry
2022/07/11 13:43:42 object contains unparsed element: map[compute:map[aggregation:count] data_source:ci_tests group_by:[map[facet:error.type limit:100 sort:map[aggregation:count order:desc]]] indexes:[*] name:query1 search:map[query:@test.status:fail @ci.pipeline.name:"Test on PR"]]
2022/07/11 13:43:42 WARN: Fail read resource from provider, wait 300ms before retry
2022/07/11 13:43:42 object contains unparsed element: map[compute:map[aggregation:count] data_source:ci_tests group_by:[map[facet:error.type limit:100 sort:map[aggregation:count order:desc]]] indexes:[*] name:query1 search:map[query:@test.status:fail @ci.pipeline.name:"Test on PR"]]
2022/07/11 13:43:42 WARN: Fail read resource from provider, wait 300ms before retry
2022/07/11 13:43:43 object contains unparsed element: map[compute:map[aggregation:count] data_source:ci_tests group_by:[map[facet:error.type limit:100 sort:map[aggregation:count order:desc]]] indexes:[*] name:query1 search:map[query:@test.status:fail @ci.pipeline.name:"Test on PR"]]
2022/07/11 13:43:43 WARN: Fail read resource from provider, wait 300ms before retry
2022/07/11 13:43:44 object contains unparsed element: map[compute:map[aggregation:count] data_source:ci_tests group_by:[map[facet:error.type limit:100 sort:map[aggregation:count order:desc]]] indexes:[*] name:query1 search:map[query:@test.status:fail @ci.pipeline.name:"Test on PR"]]
2022/07/11 13:43:44 WARN: Fail read resource from provider, wait 300ms before retry
2022/07/11 13:43:44 Fail read resource from provider, trying import command
2022/07/11 13:43:44 Filtered number of resources for service dashboard: 1
2022/07/11 13:43:44 datadog Connecting....
2022/07/11 13:43:44 datadog save dashboard
2022/07/11 13:43:44 datadog save tfstate for dashboard
olenirwing commented 2 years ago

I have the same problem: 2022/07/11 16:43:35 object contains unparsed element: map[columns:[map[field:status_line width:auto]] query:map[data_source:logs_pattern_stream group_by:[map[facet:service]] indexes:[] query_string:service:my-service status:error] response_format:event_list]

henrycpainter commented 2 years ago

Same here. After manually creating a dashboard I've got this error importing via terraformer:

2022/08/08 13:40:43 WARN: Fail read resource from provider, wait 300ms before retry
2022/08/08 13:40:44 object contains unparsed element: map[compute:map[aggregation:count] data_source:ci_tests group_by:[map[facet:@test.name limit:500 sort:map[aggregation:count order:desc]]] indexes:[*] name:query1 search:map[query:test_level:test @ci.pipeline.name:"My Pipeline"]]

Versions Datadog provider 3.14.0 Terraform v1.2.5 Terraformer version v0.8.21 Command run: terraformer import datadog --resources=dashboard --filter=dashboard="$DASHBOARD_NAME"

stiliajohny commented 2 years ago

@skarimo Out of interest ( and without looking in the PR :D ), what seems to be the fix?

skarimo commented 2 years ago

@stiliajohny Just looked into this a bit more and my pr does not actually fix the issue. The issue is with missing datasource type here: https://github.com/DataDog/datadog-api-client-go/blob/master/api/datadogV1/model_formula_and_function_events_data_source.go#L16-L25

We are missing data_source:ci_tests. I had previously added data_source:logs_pattern_stream but missed the other one. Since the client doesn't recognize this type it is unable to parse the response properly. I will put up a fix and update the client again.

skarimo commented 2 years ago

Hi all, quick update, we released the new version of Datadog terraform provider v3.15.0 which uses the latest datadog-api-client-go package and it now contains the support for both data_source:ci_tests and data_source:ci_tests enums. Thanks for the report.

github-actions[bot] commented 1 year ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days.

stiliajohny commented 1 year ago

@skarimo thanks for the update and the work on this ! Closing as done