search

orkes-io / orkes-queues

High performance queuing recipe for Netflix Conductor
Other
17 stars 10 forks source link

Orkes queues not perfoming well #4

Open sks1995 opened 1 year ago

sks1995 commented 1 year ago

Describe the bug We are trying to use orkes queues in netflix conductor As suggested in doc, changed conductor.queue.type=redis_standalone

On comparing it with dyno queues, orkes queues seemed to be under performing. I am trying to execute 3 tasks sequentially. Overall time taken by orkes queues is around 3s mostly, while for dyno queues, it is around 1.2s There is huge delay b/w task being scheduled v/s task moved to in progress. The worker in running at same frequency in both the cases.

Am I missing any configuration ? Can both dyno queues and orkes queues dependencies be included together in classpath ?

Expected behavior Expected orkes queues to be equally/more performant.

Device/browser

Additional context Add any other context about the problem here.

sks1995 commented 1 year ago

@v1r3n Can you please assist ? We are in middle of load testing.

v1r3n commented 1 year ago

Hi @sks1995 dyno-queues and orkes-queues can be in the same path - should be fine.

I would like to know more about your setup. Can we connect and go over the setup?

pmchung commented 1 year ago

I think there are artificial delays added such as, most notably the second snippet: https://github.com/orkes-io/orkes-queues/blob/65472b5cdc0a0c48cd6e6230b050e2281d68e5b7/orkes-conductor-queues/src/main/java/io/orkes/conductor/queue/dao/BaseRedisQueueDAO.java#L103-L107 https://github.com/orkes-io/orkes-queues/blob/65472b5cdc0a0c48cd6e6230b050e2281d68e5b7/orkes-redis-queues/src/main/java/io/orkes/conductor/mq/redis/QueueMonitor.java#L159-L166