NEW_RELIC_MSSQL_ENABLE_BUFFER_METRICS or NEW_RELIC_MSSQL_ENABLE_RESERVE_METRICS not respected

alessard-trackforce commented 1 month ago

Description

When using NEW_RELIC_MSSQL_ENABLE_RESERVE_METRICS: false or NEW_RELIC_MSSQL_ENABLE_BUFFER_METRICS: false in the ansible environment, the mssql-config.yml is not properly populated with these configurations (it is excluded entirely from the .yml file).

Steps to Reproduce

Include in your playbook like below:

    - role: newrelic.newrelic_install
      vars:
        targets:
          - infrastructure
          - mssql
      when: newrelic_installed
      tags: newrelic
  environment:
    NEW_RELIC_API_KEY: "{{ newrelic_api_key }}"
    NEW_RELIC_ACCOUNT_ID: "{{ newrelic_account_id }}"
    NEW_RELIC_REGION: "US"
    NEW_RELIC_MSSQL_DB_HOSTNAME: "{{ inventory_hostname }}"
    NEW_RELIC_MSSQL_DB_PORT: "{{ sql_port }}"
    NEW_RELIC_MSSQL_DB_USERNAME: "{{ newrelic_mssql_db_user }}"
    NEW_RELIC_MSSQL_DB_PASSWORD: "{{ newrelic_mssql_db_password }}"
    NEW_RELIC_MSSQL_ENABLE_BUFFER_METRICS: false
    NEW_RELIC_MSSQL_ENABLE_RESERVE_METRICS: false

Run ansible playbook / role.

Inspect file produced at C:\Program Files\New Relic\newrelic-infra\integrations.d\mssql-config.yml. It should look like:

integrations:
- name: nri-mssql
env: 
  HOSTNAME: {{ inventory_hostname }}
  USERNAME: {{ newrelic_mssql_db_user }}
  PASSWORD: {{ newrelic_mssql_db_password }}
  PORT: {{ sql_port }}
inventory_source: config/mssql
interval: 15

The contents are populated with all set variables, except that the two configurations are excluded from the resulting file.

Expected Behavior

I expect a configuration file (C:\Program Files\New Relic\newrelic-infra\integrations.d\mssql-config.yml) like the below to be generated:

integrations:
  - name: nri-mssql
    env: 
      HOSTNAME: {{ inventory_hostname }}
      USERNAME: {{ newrelic_mssql_db_user }}
      PASSWORD: {{ newrelic_mssql_db_password }}
      PORT: {{ sql_port }}
      ENABLE_BUFFER_METRICS: false
      ENABLE_RESERVE_METRICS: false
    inventory_source: config/mssql
    interval: 15

If I manually configure this file and save, the expected result works with buffer metrics and reserve metrics disabled.

Relevant Logs / Console output

changed: [hostname] => {"ansible_async_watchdog_pid": 5568, "ansible_job_id": "j474687378883.2592", "changed": true, "cmd": "[Net.ServicePointManager]::SecurityProtocol = 'tls12, tls'\n(New-Object System.Net.WebClient).DownloadFile(\"https://download.newrelic.com/install/newrelic-cli/scripts/install.ps1\", \"$env:TEMP\\install.ps1\")\n& PowerShell.exe -ExecutionPolicy Bypass -File $env:TEMP\\install.ps1\n& \"C:\\Program Files\\New Relic\\New Relic CLI\\newrelic.exe\" install -y -n infrastructure-agent-installer,mssql-server-integration-installer --tag nr_deployed_by:ansible-install 2>&1", "delta": "0:02:59.747460", "end": "2024-06-03 16:42:55.382774", "finished": 1, "rc": 0, "results_file": "C:\\Users\\redacted\\.ansible_async\\j474687378883.2592", "start": "2024-06-03 16:39:55.635314", "started": 1, "stderr": "", "stderr_lines": [], "stdout": "\n _   _                 ____      _ _\n| \\ | | _____      __ |  _ \\ ___| (_) ___\n|  \\| |/ _ \\ \\ /\\ / / | |_) / _ | | |/ __|\n| |\\  |  __/\\ V  V /  |  _ |  __| | | (__\n|_| \\_|\\___| \\_/\\_/   |_| \\_\\___|_|_|\\___|\n\nWelcome to New Relic. Let's set up full stack observability for your environment.\nOur Data Privacy Notice: https://newrelic.com/termsandconditions/services-notices\n\t\n   Connected \n\n\nInstalling New Relic Infrastructure Agent\n\n   Installed \n\n   Installed \n\n  New Relic installation complete \n\n  --------------------\n  Installation Summary\n\n  ?  Infrastructure Agent  (installed)  \n  ?  Microsoft/SqlServer Integration  (installed)  \n\n  View your data at the link below:\n  ?  https://onenr.io/redacted\n\n  --------------------\n\n", "stdout_lines": ["", " _   _                 ____      _ _", "| \\ | | _____      __ |  _ \\ ___| (_) ___", "|  \\| |/ _ \\ \\ /\\ / / | |_) / _ | | |/ __|", "| |\\  |  __/\\ V  V /  |  _ |  __| | | (__", "|_| \\_|\\___| \\_/\\_/   |_| \\_\\___|_|_|\\___|", "", "Welcome to New Relic. Let's set up full stack observability for your environment.", "Our Data Privacy Notice: https://newrelic.com/termsandconditions/services-notices", "\t", "   Connected ", "", "", "Installing New Relic Infrastructure Agent", "", "   Installed ", "", "   Installed ", "", "  New Relic installation complete ", "", "  --------------------", "  Installation Summary", "", "  ?  Infrastructure Agent  (installed)  ", "  ?  Microsoft/SqlServer Integration  (installed)  ", "", "  View your data at the link below:", "  ?  https://onenr.io/redacted", "", "  --------------------", ""]}

Your Environment

Windows Server 2019 with SQL Enterprise ansible [core 2.16.1] newrelic/newrelic_install latest

Additional context

alessard-trackforce commented 1 month ago

I should add, the newrelic/ansible-install essentially is running this powershell script, shown below:

$p = New-Object System.Security.Principal.WindowsPrincipal([System.Security.Principal.WindowsIdentity]::GetCurrent())
if (!$p.IsInRole([System.Security.Principal.WindowsBuiltInRole]::Administrator)) {
    throw 'This script requires admin privileges to run and the current Windows PowerShell session is not running as Administrator. Start Windows PowerShell by using the Run as Administrator option, and then try running the script again.'
}
[Net.ServicePointManager]::SecurityProtocol = 'tls12, tls';
$WebClient = New-Object System.Net.WebClient
if ($env:HTTPS_PROXY) {
    $WebClient.Proxy = New-Object System.Net.WebProxy($env:HTTPS_PROXY, $true)
}
$version = $null
try {
    $version = $WebClient.DownloadString("https://download.newrelic.com/install/newrelic-cli/currentVersion.txt").Trim();
    $WebClient.DownloadFile("https://download.newrelic.com/install/newrelic-cli/${version}/NewRelicCLIInstaller.msi", "$env:TEMP\NewRelicCLIInstaller.msi");
}
catch {
    Write-Output "`nCould not download the New Relic CLI installer.`n`nCheck your firewall settings. If you are using a proxy, make sure that you are able to access https://download.newrelic.com and that you have set the HTTPS_PROXY environment variable with your full proxy URL.`n"
    throw
}
msiexec.exe /qn /i $env:TEMP\NewRelicCLIInstaller.msi | Out-Null;

So I expect that ENABLE_BUFFER_METRICS and ENABLE_RESERVE_METRICS should be available options within the MSI.

pranav-new-relic commented 1 month ago

Thank you for reporting this, @alessard-trackforce. Upon brief investigation, it looks like this might not be caused by the MSI, but is instead, a glitch we could be having with the procedure that is used by ansible (which we have in open-install-library), which could be missing out on adding this attribute to mssql-config.yml.

https://github.com/newrelic/open-install-library/blob/10f52392564cb8b4c71d2979bf3400f47779d77f/recipes/newrelic/infrastructure/ohi/sql/ms-sql.yml#L174-L182

This block of code is preceeded by

https://github.com/newrelic/open-install-library/blob/10f52392564cb8b4c71d2979bf3400f47779d77f/recipes/newrelic/infrastructure/ohi/sql/ms-sql.yml#L144-L145

Though this needs further investigation, it could be probable that the condition evaluating the existence of both of these attributes in the environment variables is not entirely right, causing this behaviour (not populating the file with these attributes when the right condition is met).

Our team shall be investigating into this issue based on the current priority queue, since this scenario would need to be better comprehended by the team to make sure an apt code change is published, and changes made do not cause disruption to the working state of this recipe. Thanks for creating this issue, again!

alessard-trackforce commented 1 month ago

Thank you @pranav-new-relic - After looking at the code blocks you posted, and seeing that it is expecting 0 (instead of false), I was able to devise a work-around. If I use the following in my playbook, I get the expected result with the metrics disabled.

    - role: newrelic.newrelic_install
      vars:
        targets:
          - infrastructure
          - mssql
      when: newrelic_installed
      tags: newrelic
  environment:
    NEW_RELIC_API_KEY: "{{ newrelic_api_key }}"
    NEW_RELIC_ACCOUNT_ID: "{{ newrelic_account_id }}"
    NEW_RELIC_REGION: "US"
    NEW_RELIC_MSSQL_DB_HOSTNAME: "{{ inventory_hostname }}"
    NEW_RELIC_MSSQL_DB_PORT: "{{ sql_port }}"
    NEW_RELIC_MSSQL_DB_USERNAME: "{{ newrelic_mssql_db_user }}"
    NEW_RELIC_MSSQL_DB_PASSWORD: "{{ newrelic_mssql_db_password }}"
    NEW_RELIC_MSSQL_ENABLE_BUFFER_METRICS: 0
    NEW_RELIC_MSSQL_ENABLE_RESERVE_METRICS: 0

pranav-new-relic commented 1 month ago

Yeah, I did want to suggest this as a workaround after a little more lookup, @alessard-trackforce. Good to know it works for you now with this workaround :) however, we shall embark on having this fixed sometime, based on our current queue of tasks.

newrelic / ansible-install