Resource `newrelic_log_parsing_rule`: Issue with NRQL/query field not present

[cl0udf0x]

Hi,

Seeing the following issue. In a nutshell the query field is not being populated when creating a parsing rule using resource described below.

Terraform Version

Terraform v1.2.6

- Installed newrelic/newrelic v3.35.1
- Installing hashicorp/aws v5.47.0...
- Installed hashicorp/aws v5.47.0 (signed by HashiCorp)
- Installing hashicorp/tls v4.0.5...
- Installed hashicorp/tls v4.0.5 (signed by HashiCorp)
- Installing hashicorp/random v3.6.1...
- Installed hashicorp/random v3.6.1 (signed by HashiCorp)
- Installing hashicorp/template v2.2.0...
- Installed hashicorp/template v2.2.0 (signed by HashiCorp)

Affected Resource(s)

newrelic_log_parsing_rule

resource "newrelic_log_parsing_rule" "application_retry"{
  for_each     = var.env_mapping[var.environment]["application"]["front"]
  name         = "app-parsing-rule-retry-${each.value.application}-${each.value.environment}"
  attribute    = "message"
  enabled      = true
  grok         = <<-EOT
  \[%%{YEAR}-%%{MONTHNUM}-%%{MONTHDAY} %%{HOUR}:%%{MINUTE}:%%{SECOND},%%{INT}\: %%{LOGLEVEL:log_level}/%%{WORD:process_name}-%%{WORD}\] Task %%{DATA:task_name}\[%%{UUID:task_id}\] %%{WORD:task_status}: %%{GREEDYDATA:message}
  EOT
  lucene       = "logtype:linux_messages"
  #nrql        = "select * from Log where aws.logGroup = '${each.value.application}-${each.value.environment}' AND aws.logStream LIKE '%services-celery-default-queue/${each.value.application}-${each.value.environment}-services-celery-default-queue/%' AND message LIKE '%retry%'"
  #nrql        = "select * from log where aws.logGroup = 'app-qa' AND logStream LIKE '%services-celery-default-queue/app-qa-services-celery-default-queue/%' AND message LIKE '%retry%'"
  nrql         = "SELECT * FROM Log where aws.logGroup = 'app-qa'"
}
}

Actual Behavior

The deployment completes successfully, however the query field is missing. You can see in the hcl that different nrql have been tested and then commented out.

Expected Behavior

The query field should be populated with the the value of the nrql variable.

Steps to Reproduce

1. terraform apply

References

Similar issue reported here: https://github.com/newrelic/terraform-provider-newrelic/issues/2484

[pranav-new-relic]

Hi @cl0udf0x, thank you for reporting this, we'll take a look soon based on our task queue.

[shashank-reddy-nr]

Hi @cl0udf0x, I have been able to reproduce the issue with your configuration. I found that if you modify your query to

SELECT * FROM Log WHERE aws.logGroup = 'app-qa'

(replacing where to WHERE) will work in your case(the query of the log parsing rule is showing in the UI as well).

The right syntax for NRDB queries that go with log parsing rules should be a SELECT * FROM Log WHERE .... and the remaining of the query after WHERE is picked up by the UI (i.e., SELECT * FROM Log WHERE is expected to remain constant in queries of all log parsing rules).

We have been in conversation with the team which maintains logs to understand this issue better, and if an actual error should be thrown by the API so we propagate it into the Terraform Provider - we shall check with them again to make sure the experience with such erroneous behaviour can be made better, if this is deemed a valid occurrence.

Please let me know if it is working for you as well with the above change. Thanks

[cl0udf0x]

Hi @shashank-reddy-nr. Replacing where with WHERE has resolved the issue. Thanks for your help.

On a seperate note I have also noticed that using the aws prefix seems to break the matching log feature. Happy to open this under a new issue if you think its appropriate.

For example if the query has the following Log WHERE aws.logGroup then I get this result in the New Relic gui:

If the aws prefix is absent then matching logs are present:

[shashank-reddy-nr]

Hi @cl0udf0x , This looks like something related to the log parsing feature entirely, not specific to Terraform, so we'll forward this feedback to the logging team to take a look; however, we shall close this issue so our team's emphasis remains on currently active issues.