Foresee a warning message "Serious impact on 'Wait for previous destination' behaviour" when enabling "Include Filter/Transformer" in the Advanced Queue Settings screen

[rbeckman-nextgen]

When you enable the option "Include Filter/Transformer" in the Advanced Queue Settings screen the filter- and transformers are not executed in the right sequence when you have multiple destinations. (see attached example channel).

e.g. the filter of Destination 2 is executed BEFORE the transformer of Destination 1 ... which means that you can't use channelMap variables set in one destination in another destination.

2016-11-03 15:36:19,024 [Preprocessor JavaScript Task on testDestinationOrder (870c4ce8-1098-4912-9399-5d556d632d27) < pool-6-thread-8] preprocessor: Preprocessor script 2016-11-03 15:36:19,356 [Postprocessor JavaScript Task on testDestinationOrder (870c4ce8-1098-4912-9399-5d556d632d27) < pool-6-thread-8] postprocessor: Postprocessor script 2016-11-03 15:36:20,657 [Destination Filter/Transformer JavaScript Task on testDestinationOrder (870c4ce8-1098-4912-9399-5d556d632d27), Destination 1 (1) < pool-6-thread-8] filter: Filter of Destination 1 2016-11-03 15:36:20,658 [Destination Filter/Transformer JavaScript Task on testDestinationOrder (870c4ce8-1098-4912-9399-5d556d632d27), Destination 2 (2) < pool-6-thread-9] filter: Filter of Destination 2 2016-11-03 15:36:20,770 [Destination Filter/Transformer JavaScript Task on testDestinationOrder (870c4ce8-1098-4912-9399-5d556d632d27), Destination 1 (1) < pool-6-thread-8] transformer: Transformer of Destination 1 2016-11-03 15:36:20,774 [Destination Filter/Transformer JavaScript Task on testDestinationOrder (870c4ce8-1098-4912-9399-5d556d632d27), Destination 2 (2) < pool-6-thread-9] transformer: Transformer of Destination 2 2016-11-03 15:36:20,886 [JavaScript Writer JavaScript Task on testDestinationOrder (870c4ce8-1098-4912-9399-5d556d632d27), Destination 1 (1) < pool-6-thread-9] js-connector: Destination 1 send 2016-11-03 15:36:20,891 [JavaScript Writer JavaScript Task on testDestinationOrder (870c4ce8-1098-4912-9399-5d556d632d27), Destination 2 (2) < pool-6-thread-8] js-connector: Destination 2 send

This has a serious impact on the "Wait for previous destination" functionality !

Please investigate ... and decide what's the best solution: bugfix this ... or foresee a Warning message when this option is activated ...

Thanks a lot !

Nico

Imported Issue. Original Details: Jira Issue Key: MIRTH-4073 Reporter: nicovn Created: 2016-11-03T07:49:37.000-0700

[rbeckman-nextgen]

example channel that logs filter and transformer steps/order ...

Imported Comment. Original Details: Author: nicovn Created: 2016-11-03T07:51:25.000-0700

[rbeckman-nextgen]

This is intended and expected, and I don't think it warrants a warning message. If you have queuing enabled on a destination, you should already not expect anything in that destination to be synchronous or ordered with respect to any other destination. And the "Include Filter/Transformer" option is merely an extension of that, making the filter/transformer script part of the asynchronous queue.

Imported Comment. Original Details: Author: narupley Created: 2016-11-03T08:27:56.000-0700

[rbeckman-nextgen]

Hi Nick,

Thanks for your feedback.

I undestand.

However it's a bit confusing that pre- and postprocessor are executed before the actual destinations ... so quite useless when queuing is enabled ...

Imported Comment. Original Details: Author: nicovn Created: 2016-11-04T00:56:14.000-0700

[rbeckman-nextgen]

That's not quite correct... Destination queues are completely asynchronous with respect to the main message processing thread (which includes the postprocessor script). So it's not correct to say that the postprocessor executes "before the actual destination". In truth, the destination dispatching may happen before, after, or at the same time. It all depends on what the channel is doing and how long it takes. For example you can put a Thread.sleep in the postprocessor and see that the queued destination gets dispatched before the postprocessor finishes. Or you can move the Thread.sleep to the destination dispatcher (like a JavaScript Writer or in the response transformer) and observe the opposite happen.

It's also not true that the postprocessor is "useless" when destination queuing is enabled. The postprocessor can be used in a variety of ways. You can construct and return a custom response to the originating system. You can update a row in some external database, etc. The only difference is that if the destination gets queued, you won't be able to synchronously access the response from that destination in the postprocessor script. But if you need to do something after a destination dispatches and you also have queuing enabled, that's all still possible. Just use the response transformer for that.

Imported Comment. Original Details: Author: narupley Created: 2016-11-04T11:53:21.000-0700