search

pulumi / pulumi-newrelic

An New Relic Pulumi resource package, providing multi-language access to New Relic
Apache License 2.0
16 stars 6 forks source link

how can I assign the value of guid in newrelic.synthetics.ScriptMonitor to a static variable? #817

Closed fsdrw08 closed 1 month ago

fsdrw08 commented 1 month ago

Describe what happened

I need to refer newrelic.synthetics.ScriptMonitor's guid, then render this guid into a list of nrqlAlertConditions nrql query my config:

config:
  nrCertMonitoring:nrqlAlertConditions:
    # daysNearlyExpiration (1)
    - required:
        name: SRE_Monitor_Certificates-alert_condition_expiring_1-prod
        nrql:
          query: SELECT latest(custom.nearlyExpiringCertsLength) FROM SyntheticCheck WHERE entityGuid = '{}'
        # warning:
        critical:
          threshold: 1
          operator: above_or_equals
          threshold_duration: 60
          threshold_occurrences: AT_LEAST_ONCE
      optional:
        type: static
        enabled: true
    # daysBeforeExpiration (30)
    - required:
        name: SRE_Monitor_Certificates-alert_condition_expiring_2-prod
        nrql:
          query: SELECT latest(custom.expiringCertsLength) FROM SyntheticCheck WHERE entityGuid = '{}'
        warning:
          threshold: 1
          operator: above_or_equals
          threshold_duration: 60
          threshold_occurrences: AT_LEAST_ONCE
        # critical:
      optional:
        type: static
        enabled: true

I need to update this config by below code

# update nrql alert condition's query, put script monitor id into the query
nrql_alert_conditions_config = config.require_object("nrqlAlertConditions")
nrql_alert_conditions_config_updated = []
for nrqlAlertCondition in nrql_alert_conditions_config:
    nrqlAlertCondition_updated = nrqlAlertCondition.copy()
    query: str = str(nrqlAlertCondition["required"]["nrql"]["query"])
    # in order to put script monitor guid into the query, we have to use below method to present the query string
    nrqlAlertCondition_updated["required"]["nrql"]["query"] = script_monitor.guid.apply(lambda guid: query.format(guid))

    nrql_alert_conditions_config_updated.append(nrqlAlertCondition_updated)

It doesnt work because the variable assigned to nrqlAlertCondition_updated["required"]["nrql"]["query"] is a async object, it makes the result of the query will be the same one.

Now I would like to set the guid in newrelic.synthetics.ScriptMonitor to a static variable, how?

Sample program

see above

Log output

No response

Affected Resource(s)

No response

Output of pulumi about

CLI
Version      3.122.0
Go Version   go1.22.4
Go Compiler  gc

Plugins
KIND      NAME      VERSION
resource  newrelic  5.25.3
language  python    unknown

Host
OS       Microsoft Windows 11 Enterprise
Version  10.0.22621 Build 22621
Arch     x86_64

This project is written in python: executable='C:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\sre-iac-6JBwHF1h-py3.11\Scripts\python.exe' version='3.11.9'

Backend
Name           DESKTOP-IN2K25I
URL            azblob://pulumistate/nrcertmonitoring
User           AzureAD\xxx
Organizations
Token type     personal

Dependencies:
NAME                  VERSION
black                 23.12.1
flakehell             0.9.0
pip                   23.3.1
pulumi_newrelic       5.25.3
pytest                7.4.2
requests              2.29.0
serko-product-config  1.6.10
yamllint              1.32.0

Pulumi locates its logs in C:\Users\xxx\AppData\Local\Temp by default
warning: Failed to get information about the current stack: No current stack

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

iwahbe commented 1 month ago

Hi @fsdrw08 Can you show me an example of what you are trying to accomplish? It's not possible to go from an Output[T] to a T, but I take a look at what your doing and see if I can provide a working solution.

fsdrw08 commented 1 month ago

Here is the pulumi arg config

config:
  nrCertMonitoring:scriptMonitor:
    name: SRE_Monitor_Certificates
    status: ENABLED
    locations_publics:
      - AWS_AP_SOUTHEAST_2
    period: EVERY_DAY
    type: SCRIPT_API
    runtime_type: NODE_API
    runtime_type_version: "16.10"
    script_language: JAVASCRIPT
  nrCertMonitoring:nrqlAlertConditions:
    # daysNearlyExpiration (1)
    - required:
        name: SRE_Monitor_Certificates-alert_condition_expiring_1-prod
        nrql:
          # need to render the script monitor's guid into query, 
          query: SELECT latest(custom.nearlyExpiringCertsLength) FROM SyntheticCheck WHERE entityGuid = '{}'
        # warning:
        critical:
          threshold: 1
          operator: above_or_equals
          threshold_duration: 60
          threshold_occurrences: AT_LEAST_ONCE
      optional:
        type: static
        enabled: true
    # daysBeforeExpiration (30)
    - required:
        name: SRE_Monitor_Certificates-alert_condition_expiring_2-prod
        nrql:
          query: SELECT latest(custom.expiringCertsLength) FROM SyntheticCheck WHERE entityGuid = '{}'
        warning:
          threshold: 1
          operator: above_or_equals
          threshold_duration: 60
          threshold_occurrences: AT_LEAST_ONCE
        # critical:
      optional:
        type: static
        enabled: true

We need to create newrelic.synthetics.ScriptMonitor, then get ScriptMonitor's guid, render the guid into the list object's attribute config.nrqlAlertConditions[*].required.nrql, then pass the rendered arg into newrelic.NrqlAlertCondition

fsdrw08 commented 1 month ago

I understand now, new relic script_monitor's guid is a "output" of pulumi.
Regarding to Creating new output values, "output" are asynchronous, pulumi treat it as an unknown value

In pulumi, we can only express this output value by the "apply" method.(the apply method will return a output object)
Which means: everything related to this "output", they all need to included under the apply method. You have to make some functions, then bind them together as a lambda, put it under "apply".

Here is my solution:

config = Config()
# create script monitor
script_monitor = newrelic.synthetics.ScriptMonitor(
    f'{(config.require_object("scriptMonitor").get("name"))}_{get_stack()}',
    name=f'{(config.require_object("scriptMonitor").get("name"))}_{get_stack()}',
    # https://docs.newrelic.com/docs/synthetics/synthetic-monitoring/administration/synthetic-public-minion-ips/#location
    locations_publics=config.require_object("scriptMonitor").get("locations_publics"),
    period=config.require_object("scriptMonitor").get("period"),
    type=config.require_object("scriptMonitor").get("type"),
    runtime_type=config.require_object("scriptMonitor").get("runtime_type"),
    runtime_type_version=config.require_object("scriptMonitor").get("runtime_type_version"),
    script_language=config.require_object("scriptMonitor").get("script_language"),
    script=nrScript.getFullNRScript(),
    status=config.require_object("scriptMonitor").get("status"),
    tags=tags,
)

# prepare a function, to update nrql alert condition's query, put script monitor id into the query
def update_nrql_alert_conditions_config(nrql_alert_conditions_config: list, script_monitor_guid) -> list:
    nrql_alert_conditions_config_updated = []
    for nrqlAlertCondition in nrql_alert_conditions_config:
        nrqlAlertCondition_updated = nrqlAlertCondition.copy()
        query: str = str(nrqlAlertCondition["required"]["nrql"]["query"])
        # in order to put script monitor guid into the query, we have to use below method to present the query string
        nrqlAlertCondition_updated["required"]["nrql"]["query"] = query.format(script_monitor_guid)

        nrql_alert_conditions_config_updated.append(nrqlAlertCondition_updated)
    return nrql_alert_conditions_config_updated

nrql_alert_conditions_config = config.require_object("nrqlAlertConditions")
# We need to get script_monitor's guid, render the guid into alert_condition's query under the list object nrql_alert_conditions_config
# which get from pulumi config "nrqlAlertConditions",
# then pass the updated(rendered) object as a argument into the custom resource component (aka terraform's module).
#
# new relic script_monitor's guid is a "output" of pulumi,
# regarding to https://www.pulumi.com/docs/concepts/inputs-outputs/apply/#creating-new-output-values
# "output" are asynchronous, it's an unknown value, in pulumi, we can only express this output value by the "apply" method.(the apply method will return a output object)
# which means: everything related to this "output", they all need to included under the apply method.
# you have to make some functions, then bind them together as a lambda, put it under "apply"
# ref: https://www.pulumi.com/docs/concepts/inputs-outputs/all/
pulumi.Output.all(guid=script_monitor.guid).apply(
    lambda args: NewrelicAlertBundle(
        project_name=config.require_object("scriptMonitor").get("name"),
        alert_policy_args=config.require_object("alertPolicy"),
        nrql_alert_conditions_args=update_nrql_alert_conditions_config(
            nrql_alert_conditions_config=nrql_alert_conditions_config, script_monitor_guid=args["guid"]
        ),
        notice_non_urgent_args=config.get_object("notice_non_urgent"),
        notice_urgent_args=config.get_object("notice_urgent"),
        opts=pulumi.ResourceOptions(depends_on=script_monitor),
    )
)

you dont need to care about how does this "NewrelicAlertBundle" work,
just keep in mind: you have to prepare some functions (which terraform is no needed), then bind them together as a lambda, put it under "apply" in this case, I had to prepare 2 function/class:

  1. the update_nrql_alert_conditions_config function to update config,
  2. a pulumi resource component class NewrelicAlertBundle which will create multi alert conditions what I need

and pass this function in to the class

Compare to terraform, pulumi is so counterintuitive, and the learning curve is high

fsdrw08 commented 1 month ago

one more thing, with in the code above, while running pulumi preview, the preview will not show the newrelic alert condition related resource in the plan

pulumi up --stack test
Previewing update (test):
     Type                                  Name                                     Plan
 +   pulumi:pulumi:Stack                   nrCertMonitoring-test                    create
 +   ├─ newrelic:synthetics:ScriptMonitor  SRE_Monitor_Certificates_test            create
 +   └─ newrelic:index:OneDashboard        SRE_Monitor_Certificates-dashboard-test  create                                                                                                                                                 
Resources:
    + 3 to create

Do you want to perform this update? yes
Updating (test):
     Type                                      Name                                                        Status
 +   pulumi:pulumi:Stack                       nrCertMonitoring-test                                       created (18s)
 +   ├─ newrelic:synthetics:ScriptMonitor      SRE_Monitor_Certificates_test                               created (3s)
 +   ├─ custom:resource:NewRelicAlerts         SRE_Monitor_Certificates                                    created
 +   │  ├─ newrelic:index:AlertPolicy          SRE_Monitor_Certificates-alert_policy                       created (1s)
 +   │  ├─ newrelic:index:NotificationChannel  SRE_Monitor_Certificates-notification_channel_non_urgent-0  created (2s)
 +   │  ├─ newrelic:index:NrqlAlertCondition   SRE_Monitor_Certificates-nrql_alert_condition-0             created (2s)
 +   │  └─ newrelic:index:Workflow             SRE_Monitor_Certificates-workflow_non_urgent                created (1s)
 +   └─ newrelic:index:OneDashboard            SRE_Monitor_Certificates-dashboard-test                     created (2s)
Resources:
    + 8 created

Duration: 20s
iwahbe commented 1 month ago

You won't see resources in preview since you created then in an .apply. I would try:

...

# prepare a function, to update nrql alert condition's query, put script monitor id into the query
def update_nrql_alert_conditions_config(nrql_alert_conditions_config: List[Any], script_monitor_guid: str) -> List[str]:
    nrql_alert_conditions_config_updated = []
    for nrqlAlertCondition in nrql_alert_conditions_config:
        nrqlAlertCondition_updated = nrqlAlertCondition.copy()
        query: str = str(nrqlAlertCondition["required"]["nrql"]["query"])
        # in order to put script monitor guid into the query, we have to use below method to present the query string
        nrqlAlertCondition_updated["required"]["nrql"]["query"] = query.format(script_monitor_guid)

        nrql_alert_conditions_config_updated.append(nrqlAlertCondition_updated)
    return nrql_alert_conditions_config_updated

nrql_alert_conditions_config = config.require_object("nrqlAlertConditions")
# We need to get script_monitor's guid, render the guid into alert_condition's query under the list object nrql_alert_conditions_config
# which get from pulumi config "nrqlAlertConditions",
# then pass the updated(rendered) object as a argument into the custom resource component (aka terraform's module).
#
# new relic script_monitor's guid is a "output" of pulumi,
# regarding to https://www.pulumi.com/docs/concepts/inputs-outputs/apply/#creating-new-output-values
# "output" are asynchronous, it's an unknown value, in pulumi, we can only express this output value by the "apply" method.(the apply method will return a output object)
# which means: everything related to this "output", they all need to included under the apply method.
# you have to make some functions, then bind them together as a lambda, put it under "apply"
# ref: https://www.pulumi.com/docs/concepts/inputs-outputs/all/
NewrelicAlertBundle(
    project_name=config.require_object("scriptMonitor").get("name"),
    alert_policy_args=config.require_object("alertPolicy"),
    nrql_alert_conditions_args=script_monitor.guid.apply(lambda args: \
        update_nrql_alert_conditions_config(
            nrql_alert_conditions_config=nrql_alert_conditions_config, 
            script_monitor_guid=args["guid"],
        ),
    ),
    notice_non_urgent_args=config.get_object("notice_non_urgent"),
    notice_urgent_args=config.get_object("notice_urgent"),
    opts=pulumi.ResourceOptions(depends_on=script_monitor),
)

Notice that I have moved the apply into the NewrelicAlertBundle creation so NewrelicAlertBundle takes an Output[List[str]] as the argument to nrql_alert_conditions_args. Since the resource constructor is always run, it will now show up in previews.

fsdrw08 commented 1 month ago

it doesnt work, the code related to nrql_alert_conditions_args in the class:

self.nrql_alert_conditions = []

for idx, nrql_alert_condition_args in enumerate(nrql_alert_conditions_args):
    # nrql_alert_condition_args.update({"policy_id": self.alert_policy.id})
    nrql_alert_condition = newrelic.NrqlAlertCondition(
        f"{project_name}-nrql_alert_condition-{idx}",
        # required args
        policy_id=self.alert_policy.id,
        name=nrql_alert_condition_args["required"]["name"],
        nrql=newrelic.NrqlAlertConditionNrqlArgs(query=nrql_alert_condition_args["required"]["nrql"]["query"]),
        critical=(
            None
            if nrql_alert_condition_args["required"].get("critical", None) is None
            else newrelic.NrqlAlertConditionCriticalArgs(
                operator=nrql_alert_condition_args["required"]["critical"]["operator"],
                threshold=nrql_alert_condition_args["required"]["critical"]["threshold"],
                threshold_duration=nrql_alert_condition_args["required"]["critical"]["threshold_duration"],
                threshold_occurrences=nrql_alert_condition_args["required"]["critical"][
                    "threshold_occurrences"
                ],
            )
        ),
        warning=(
            None
            if nrql_alert_condition_args["required"].get("warning", None) is None
            else newrelic.NrqlAlertConditionWarningArgs(
                operator=nrql_alert_condition_args["required"]["warning"]["operator"],
                threshold=nrql_alert_condition_args["required"]["warning"]["threshold"],
                threshold_duration=nrql_alert_condition_args["required"]["warning"]["threshold_duration"],
                threshold_occurrences=nrql_alert_condition_args["required"]["warning"]["threshold_occurrences"],
            )
        ),
        # optional args
        **nrql_alert_condition_args["optional"],
        opts=pulumi.ResourceOptions(parent=self),
    )
    self.nrql_alert_conditions.append(nrql_alert_condition)

error shows

        for idx, nrql_alert_condition_args in enumerate(nrql_alert_conditions_args):
                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
    TypeError: 'Output' object is not iterable, consider iterating the underlying value inside an 'apply'
iwahbe commented 1 month ago

Sounds like you have it figured out. Creating a resource inside an apply will yield inconstant previews, but won't have any negative effects on the result of pulumi up.

fsdrw08 commented 1 month ago

update the finally solution, I defined a function to update the dict object, it's better put the pulumi output apply process inside a function, to reduce the side effect:

def update_nrql_alert_condition_config(nrqlAlertCondition: List, script_monitor: newrelic.synthetics.ScriptMonitor):
    query: str = nrqlAlertCondition["required"]["nrql"]["query"]
    # in order to put script monitor guid into the query, we have to use below method to present the query string
    # this makes nrqlAlertCondition["required"]["nrql"]["query"] as a pulumi output object
    nrqlAlertCondition["required"]["nrql"]["query"] = script_monitor.guid.apply(lambda guid: query.format(guid))
    return nrqlAlertCondition

nrql_alert_conditions_config = config.require_object("nrqlAlertConditions")
notice_non_urgent_config = config.get_object("notice_non_urgent")
notice_urgent_config = config.get_object("notice_urgent")

NewrelicAlertBundle(
    project_name=script_monitor_config["name"],
    alert_policy_args=config.require_object("alertPolicy"),
    nrql_alert_conditions_args=map(
        lambda nrqlAlertCondition: update_nrql_alert_condition_config(nrqlAlertCondition, script_monitor),
        nrql_alert_conditions_config,
    ),
    notice_non_urgent_args=notice_non_urgent_config,
    notice_urgent_args=notice_urgent_config,
    opts=pulumi.ResourceOptions(depends_on=script_monitor),
)