String concatenation with loop iterator

[liquidharmonic]

I want to create 100,000 workers that each subscribe to a unique topic. I am using a loop on top of a pool to leverage the loop's iterator. I want add the iterators value to the topic. Unfortunately, it doesn't run. The error listed at the end disappears if I remove the concatenation of a string with var("i"). I wasn't able to find any documentation about how to do string interpolation or concatenation on. bdl. Can someone please point me in the right direction?

Thank you :)

Benchmark File

#!benchDL

make_install(rsync="/Users/eko/repos/vmq_mzbench/", exclude="deps")

loop(time = 10 sec, rate = 100 rps, iterator = "i"): 
    pool(size = 1,
         worker_type = mqtt_worker,
         worker_start = linear(10 rps)):

                connect([t(host, "127.0.0.1"),
                        t(port,1883),
                        t(client,fixed_client_id("subscriber_pool1", worker_id())),
                        t(clean_session,true),
                        t(keepalive_interval,60),
                        t(proto_version,4), t(reconnect_timeout,4)
                        ])

                set_signal(subscribe1, 1)
                wait_signal(subscribe1, 10000)
                wait(10 sec)
                subscribe("/subscriber/" + var("i"), 1)

Error

Traceback (most recent call last):
  File "./bin/mzbench", line 734, in <module>
    main()
  File "./bin/mzbench", line 717, in main
    return apply_args(globals()[cmd], args)
  File "./bin/mzbench", line 705, in apply_args
    return fun(**dict((k, get_arg(k, kwargs)) for k in argnames))
  File "./bin/mzbench", line 324, in run
    name, cloud, tags, env, no_cert_check, exclusive)
  File "./bin/mzbench", line 348, in start
    exclusive=exclusive)
  File "/Users/eko/repos/mzbench/bin/../lib/mzbench_api_client.py", line 67, in start
    script_terms = script_utils.convert(script_content, env)
  File "/Users/eko/repos/mzbench/bin/../lib/bdl_utils.py", line 12, in convert
    ast = transform(lex(contents))
  File "/Users/eko/repos/mzbench/bin/../lib/bdl_utils.py", line 174, in lex
    raise ParseError(e)
bdl_utils.ParseError: Rule 'entry' matched in its entirety, but it didn't consume all the text. The non-matching portion of the text begins with ':
_INDENT_    pool(' (line 5, column 51).

[liquidharmonic]

doh. I realized that I can use sprinf("/subscriber/%s", var("i")) to get the string I want. Sorry for the stupid question. However, I cannot nest a pool inside a loop to get the desired result of spinning 100,000 nodes and subscribing a unique topic.

[timofey-barmin]

Hi unfortunately it is not possible to use pool inside loop. Why can't you just start more workers (see size opt in pool spec) with speed you need (see worker_start in pool spec) and use var("worker_id") instead of var("i")?

[liquidharmonic]

Hi @timofey-barmin Thanks for the suggestion. The reason why I want to use the iterator is because I want to to have another pool of workers publish to the topics using var("i"). Unfortunately, the var("worker_id") because it sequentially increases

[parsifal-47]

could you please provide more details why "worker_id" is not suitable? var("i") increases as well

[liquidharmonic]

I want to isolate the subscriber and publisher for the use case where the subscriber and publisher are in different regions (Canada, US). This may not be needed if someone can reason that there is not need for the isolation.

Precisely I'd like 500 workers to each subscribe to their own unique topic /device/1, device/2, device/3, ..., device/500 and then 500 workers to each publish to the above unique topics /device/1, device/2, device/3, ..., device/500

I don't want to have a single worker subscribed to /device/1, device/2, device/3, ..., device/500. This can be achieved by having a loop inside a single worker.

[parsifal-47]

yes, but it sounds like exactly what you need, if you use "worker_id" it is going to be unique number for each worker inside a pool and they don't have any gaps, so far I see no contradiction, I'll make an example

[liquidharmonic]

sorry everyone. I didn't realize that the worker_id starts from 0 for each pool. Yes @parsifal-47 and @timofey-barmin worker_id is what I want. Closing this issue.