How does state = {} work in starlark aggregator

[snehal520]

Starlark aggregator is called for metric m1 where I add m1 to state dict, after which Starlark processor is called for metric m2 post which Starlark aggregator is called for metric m2 and value in state dict is overwritten with m2. Which I am not writing.

After first call to aggregator state = [Metric("M1", tags={"t1": "t1"}, fields={"name": "apple", "type": "fruit"}, time=1700732160000000000)]

After second call to aggregator state = [Metric("M2", tags={"t2": "t2"}, fields={"name": "carrot", "type": "veg"}, time=1700832160000000000)]

[srebhan]

@snehal520 without the corresponding starlark script (or a script reproducing the issue) it is hard to tell what is going on...

[telegraf-tiger[bot]]

Hello! I recommend posting this question in our Community Slack or Community Forums, we have a lot of talented community members there who could help answer your question more quickly. You can also learn more about Telegraf by enrolling at InfluxDB University for free!

Heads up, this issue will be automatically closed after 7 days of inactivity. Thank you!

[snehal520]

@srebhan I am trying to use starlark aggregator in telegraf. I am not able to understand how "globals["state"] = starlark.NewDict(0)" defined in https://github.com/influxdata/telegraf/blob/master/plugins/common/starlark/starlark.go is getting shared across metrics consider the below snippet

[[aggregators.starlark]]
    namepass = ["m1","m2"]
    source = '''
load("logging.star", "log")
state = {}

def add(metric):
    [log.info](http://log.info/)("Aggregator Start "+[metric.name](http://metric.name/))
    [log.info](http://log.info/)("Check in ADD  "+ str(state))
    if [metric.name](http://metric.name/) == "m1":
        if "m1" in state:
            state["m1"].append(metric)
        else:
            state["m1"] = []
    else:
        if "m2" in state:
            state["m2"].append(metric)
        else:
            state["m2"] = []

def push():
    [log.info](http://log.info/)("PUSH CALLED")
    return None

def reset():
    [log.info](http://log.info/)("RESET")
    state.clear()
'''

Output of above snippet

2023-11-24T09:34:00Z I! [aggregators.starlark] PUSH CALLED
2023-11-24T09:34:00Z I! [aggregators.starlark] RESET
2023-11-24T09:34:00Z I! [aggregators.starlark] Aggregator Start m1
2023-11-24T09:34:00Z I! [aggregators.starlark] Check in ADD  {}
2023-11-24T09:34:00Z I! [aggregators.starlark] Aggregator Start m1
2023-11-24T09:34:00Z I! [aggregators.starlark] Check in ADD  {"m1": []}
2023-11-24T09:34:00Z I! [aggregators.starlark] Aggregator Start m1
2023-11-24T09:34:00Z I! [aggregators.starlark] Check in ADD  {"m1": [Metric("m1", tags={"t1": "true"}, fields={"name": "apple", "type": "fruits"}, time=1700818440000000000)]}
2023-11-24T09:34:00Z I! [aggregators.starlark] Aggregator Start m1
2023-11-24T09:34:00Z I! [aggregators.starlark] Check in ADD  {"m1": [Metric("m1", tags={"t1": "true"}, fields={"type": "fruits", "name": "banana"}, time=1700818440000000000), Metric("m1", tags={"t1": "true"}, fields={"type": " fruits", "name": "banana"}, time=1700818440000000000)]}

In 2nd output fruit name "apple" should also be present, but it is getting replaced to only have two entries with banana.

Please give any inputs on this.

[powersj]

Let's continue this in the forums: https://community.influxdata.com/t/how-does-state-work-in-telegraf-aggregator/32256/4