Problem when using resource in a loop

[robinbowes]

Hi,

I'm using terrascript to dump existing datadog monitors into terraform code.

Here's some sample code:

import terrascript
from datadog import initialize, api
from terrascript.datadog.r import datadog_monitor

def get_all():
    """ get all monitors """
    initialize()
    response = api.Monitor.get_all()
    return response

def dump_as_tf(monitor):
    """ Dump the specified monitor as terraform code """
    kwargs = {}
    kwargs['type'] = monitor['type']
    datadog_monitor(
       monitor['name'],
       **kwargs
    )
    print(terrascript.dump())

def dump_all_as_tf():
    """ get all monitors and dump as terraform code """
    response = get_all()
    # response is an array of hashes, one hash for each monitor
    for monitor in response:
        dump_as_tf(monitor)

What I see is that the datadog_monitor resource doesn't clean up between invocations so I end up with this sort of output:

{
  "resource": {
    "datadog_monitor": {
      "name of first monitor": {
        "type": "query alert"
      }
    }
  }
}
{
  "resource": {
    "datadog_monitor": {
      "name of second monitor": {
        "type": "service check"
      },
      "name of first monitor": {
        "type": "query alert"
      }
    }
  }
}
{
  "resource": {
    "datadog_monitor": {
      "name of second monitor": {
        "type": "service check"
      },
      "name of third monitor": {
        "type": "service check"
      },
      "name of first monitor": {
        "type": "query alert"
      }
    }
  }
}

Am I doing something wrong here?

[mjuenema]

terrascript.dump() is meant to be called once, i.e. at the end of your program. I accept that this may not be clearly explained in my documentation/examples. Try to change your code accordingly and let me know how you go.

[robinbowes]

That seems to work, however I was hoping to write each resource into a separate file, ie. I would want to dump the output once for each resource.

I guess I can dump it once and loop over the resulting json, but it seems less than ideal.

[mjuenema]

Ok, I see what you are trying to achieve. That would require some kind of terrascript.reset() function which doesn't currently exists. Then one could do (in pseudo-code):

Loop over monitors:
    datadog_monitor( monitor['name'], **kwargs)
    dump = terrascript.dump()
    write_dump_to_file(dump)
    terrascript.reset()

Is that what you are trying to achieve?

[robinbowes]

Yes, exactly that.

Or maybe something like this:

Loop over monitors:
    ts = Terrascript.create()
    ts.add('datadog_monitor', monitor['name'], **kwargs)
    write_dump_to_file(ts.dump())

[mjuenema]

Have a look at my comments to https://github.com/mjuenema/python-terrascript/pull/27. Once I have rewritten the code the following will be possible which would also make your original code possible (with some minor changes).

for x in range(1,4):
    t = terrascript.Terrascript()
    t += provider('aws', region='us-east-' + x)    # us-east-1, us-east-2, us-east-3
    config = t.dump()
    # ... and write each config to separate files.

[mjuenema]

This is now possible. Have a look at tests/test_issues.py for details.

[robinbowes]

Thanks for fixing this - it's working well for me.

[mac-cooper]

@mjuenema, I too am running into this issue. After reading over the thread I am still not able to get this working as designed? Has there been an update to the code?

[mjuenema]

Hi @blackego30, there have been many changes to the code since 2018. Can you please provide some more details of the problem you are encountering. Some sample code would be very useful.

[mac-cooper]

Hello @mjuenema, initialize(**options)

Use Datadog REST API client

from datadog import api

datadog_monitors = api.Monitor.get_all()

timestamp = strftime("%Y-%m-%d_%H:%M:%S", gmtime())

print(json.dumps(datadog_monitors, indent=2, sort_keys=False))

print (timestamp) print ("Number of monitors: ", + len(str(datadog_monitors)))

with open('existing_monitors.json', 'w') as outfile: json.dump(datadog_monitors, outfile, indent=2, sort_keys=False)

Used the function below based off your test resolutions. Ideally was looking to see if I can get things flowing.

def test_issue_26():

for name in ["first", "second", "third"]:
    ts = terrascript.Terrascript()
    ts += terrascript.provider.datadog(api_key='xxxxxxxxxx', app_key='xxxxxxx')
    ts += terrascript.resource.datadog_monitor(
        name,
        name=name,
        type="metric alert",
        query="some query",
        message="a message",
        escalation_message="some message",
    )
    ts = config.dump()

    assert len(ts["resource"]["datadog_monitor"].keys()) == 1
    assert ts["resource"]["datadog_monitor"][name]["name"] == name