Refactors the HTTP target to handle batches of data in one go, adds the ability to provide a template to get those batches of data into a request body, and adds a helper function to cover the case where there is no template provided.
This involves breaking changes - the previous behaviour can be achieved by configuring a template if the data is already JSON, or a transformation + a template if not - but we don't believe there are any use cases requiring us to do so.
Refactors the http target to handle batches
Adds configuration of batch sizes for http target
Adds a templater function
Adds a helper function which creates a JSON array from the data where no template is provided
Adds a function to group data by dynamic headers where that is configured - to ensure that this feature still works
Adds tests for these features
These features also introduce a new possibility for failure in the HTTP target. The http target now expects valid JSON (previously it could be anything) - where we hit an error in either templating this data - or in the case of no template in transforming the data into a JSON array request body - these messages are treated as invalid, and will be sent to the failure target.
Refactors the HTTP target to handle batches of data in one go, adds the ability to provide a template to get those batches of data into a request body, and adds a helper function to cover the case where there is no template provided.
This involves breaking changes - the previous behaviour can be achieved by configuring a template if the data is already JSON, or a transformation + a template if not - but we don't believe there are any use cases requiring us to do so.
These features also introduce a new possibility for failure in the HTTP target. The http target now expects valid JSON (previously it could be anything) - where we hit an error in either templating this data - or in the case of no template in transforming the data into a JSON array request body - these messages are treated as invalid, and will be sent to the failure target.