search

Softlandia-Ltd / stateful-streaming-examples

Stateful streaming computations implemented with many technologies
31 stars 4 forks source link

Time Sleep #1

Open nil-andreu opened 1 year ago

nil-andreu commented 1 year ago

Hey, I was wondering which is the objective of the time.sleep in Kafka:

while True:
        msgs = generate_messages(rng, n_sensors=sensors)
        print(msgs)
        for m in msgs:
            producer.send(topic, m)

        time.sleep(1)

Is it because we want to wait one second between generating messages, or because we want to wait some time to make sure all messages are sent?

Mikkolehtimaki commented 1 year ago

Hi, thanks for the question! I just used that for debugging and evaluating the technologies. It's not necessary to use the sleep at all, but there will be a lot of data in your Kafka cluster if you leave that out. Feel free to experiment :)

nil-andreu commented 1 year ago

Hey!! What you could do is producer.flush(), to make sure that you produce all your messages before keep creating new ones.