Create a delivery service base class

[dajtxx]

Delivery services should all have similar logic so it might be worth creating a base class they can derive from what does all the work of setting up the RabbitMQ infra and leaves the delivery service itself to just process messages.

If using RabbitMQ exchanges and queues:

• Have the base class create a delivery service specific exchange that is bound to the logical timeseries exchange, and then a queue bound to the service specific exchange. This allows the different delivery services to be a bit more independent.

• Perhaps the dead letter feature can be used to allow failed messages to be retried without the same message continually ending up at the head of a queue as happens with a NAK/requeue.

• See if there is a way for the service to also put totally failed messages which are not to be retried on the DLEX so we have a record of them outside the log files but they don't keep getting retried. A new DB table might be an alternative solution to this. In either case, the base class should take a return code from the delivery class message handling function - success, retry, fail, and deal with it accordingly.

How to read the DLEX occasionally, rather than all the time? Like once an hour or so.

[dajtxx]

See also issue #36 .

This issue is using the DLEX rather than a db table as a store for undelivered messages, and still tries to deliver the message before putting it in the store.

36 suggests storing the message first and delivering later. That idea is possibly more difficult in a single process, requiring async code (with luck a thead could be used instead?) but does seem cleaner. The table or DLEX can be read through from start to finish with messages removed as they are successfully delivered, and then the cycle just starts again until the table is empty. The timing of final message delivery is also now independent of arrival for situations where there is a rate limit, such as Ubidots, so reception isn't held up by delivery rates.

If the async/threaded code can be hidden in the base class then the subclasses are still simple.

[dajtxx]

Base class is working, uses the db table idea from #36.

The db table could be more efficient if it stored the uid from the physical_devices table and then the message could be retrieved from there. This means the uid needs to be passed from the logical mapper onwards, either added to the message or replacing the message in the logical_timeseries exchange messages. Replacing means all delivery processes would need to use the new mechanism, whereas including the uid in the JSON message means delivery processes could pick it out and store it in their table.

[dajtxx]

Implemented in 35c1c38.

Note it is still worth considering just passing a message id into the logical_timeseries exchange now and having delivery services read from the physical_timeseries table. But that can wait for another time.