Question about stacking processors

[noniagriconomie]

Hi

According to https://github.com/swarrot/swarrot#using-a-stack, the order of the stacked processors is important.

Given this config: (v1.5.1)

notification_consumer:
            processor: core.component.rabbitmq.notification_processor
            middleware_stack:
                - configurator: swarrot.processor.signal_handler             
                - configurator: swarrot.processor.exception_catcher      
                - configurator: swarrot.processor.ack                               
                - configurator: swarrot.processor.max_messages           
            extras:
                max_messages: 100

What is the best order? from top (last processor executed) to bottom (first executed after our processor) I ask this question because checking my logs today, I've seen this:

[2018-04-15 01:32:55] rabbitmq.INFO: [Ack] Message #19 has been correctly ack'ed {"swarrot_processor":"ack"} {"uid":"078067949a"}
[2018-04-15 01:55:43] rabbitmq.INFO: [MaxMessages] Max messages have been reached (20) {"swarrot_processor":"max_messages"} {"uid":"078067949a"}
[2018-04-15 01:55:43] rabbitmq.INFO: [Ack] Message #20 has been correctly ack'ed {"swarrot_processor":"ack"} {"uid":"078067949a"}

and the logic would be:

[2018-04-15 01:32:55] rabbitmq.INFO: [Ack] Message #19 has been correctly ack'ed {"swarrot_processor":"ack"} {"uid":"078067949a"}
[2018-04-15 01:55:43] rabbitmq.INFO: [Ack] Message #20 has been correctly ack'ed {"swarrot_processor":"ack"} {"uid":"078067949a"}
[2018-04-15 01:55:43] rabbitmq.INFO: [MaxMessages] Max messages have been reached (20) {"swarrot_processor":"max_messages"} {"uid":"078067949a"}

Is there a good practice for this kind of stack? (exception, handling signal, max execution and ack of course)

Thank you for your time

[odolbeau]

Hi

According to your configuration, processors will be called in this order:

• signal_handler
• exception_catcher
• ack
• max_messages
• your processor
• max_messages
• ack
• exception_catcher
• signal _handler

In your case, you received message number 19. Your processor did the job, the max messages processor have been called but your still under the limit you defined. Then the ack processor did it's job. Then, you received message number 20. Your processor did the job. The max processor have been called and as you've reached you limit it did it's job (log + prevent other message). Then ack processor have been called and ack'ed the message.

I would recommend you to use this order:

• signal_handler
• max_messages
• exception_catcher
• ack

This way:

• Logs will be more understandable (first ack then max_messages & stop)
• You won't have trouble with exceptions (with your actual configuration, if the message number 20 have an exception, the max messages processor won't catch the exception, which means you will process another message)

[noniagriconomie]

hi @odolbeau,

first thank for you explanations and your time.

Just a point I am not sure to understand properly on the stack processor.

According to your configuration, processors will be called in this order: signal_handler exception_catcher ack max_messages your processor max_messages ack exception_catcher signal _handler

This means that for every message consumption, we ack 2 times (before and after our internal processor) ? we catch any ex before and after our internal processor?

I though it is just a "one pass" processor stack I am a little bit confused right now

[odolbeau]

When using a stack, your processor is decorated by others processors.

If you use only the ack processor:

• your processor is injected in the ack processor constructor
• the ack processor is called
• it calls you processor. If there is an error, the message is nacked, otherwise, it's acked.

Take a look a the implementation: https://github.com/swarrot/swarrot/blob/master/src/Swarrot/Processor/Ack/AckProcessor.php#L47-L63

This way, all processors can do some job before and / or after calling the next processor.

Not sure my explanations are really clear. :/ Does it makes sense?

[noniagriconomie]

@odolbeau thank you again for your time

what i mean is, seeing your comment (cf my previous post) i understood that the processors stack defined is call before and after our processor

that is why i asked for "clarification" after your comment, because seeing the code and the picture on the readme of the lib (https://camo.githubusercontent.com/8ac89cd415aebfb1026b2278093dbcc986b126da/68747470733a2f2f646f63732e676f6f676c652e636f6d2f64726177696e67732f642f3145615f514a486f2d3970375957386c5f62793753344e494430652d41477058527a7a6974416c59593543632f7075623f773d39363026683d373230) seems weird

Thanks

[stof]

This means that for every message consumption, we ack 2 times (before and after our internal processor) ? we catch any ex before and after our internal processor?

no, we don't ack twice. We run the AckProcessor logic in a specific place of the stack. But the AckProcessor is not acking the message both before and after calling the inner processing (actually, the AckProcessor does nothing before calling the inner processing, but it is still reached in the stack)

[noniagriconomie]

@stof ok understood Thank you both