helium / gateway-rs

The Helium Gateway
Apache License 2.0
280 stars 110 forks source link

Order of packets in queue in case of tx failure #435

Closed disk91 closed 1 year ago

disk91 commented 1 year ago

Open question :

having the frame queued in front in case of paquet router transmission failure could not be the most efficient solution. It will delay more the new frame arriving. If i understand well all the pending paquets are re-processed during 60 seconds before being killed. When service will be restored the packets will be dequeued starting by the older one. Most of them will be unexpected, in particular the one with ack / downlink req. the life time of a packet with ack or downlink could be reduced to a 200ms time windows. Uplink could be longer but i assume better loosing a packet than having a repeat that will be paid and rejected by LNS.

i think the best would be to have 2 queues, one for new packets and one for retry with a priority on the new packets.

madninja commented 1 year ago

Interesting.. and when would you retry under new packet load? The retry policy will have some complexity since retry packets will get starved out almost all the time?

madninja commented 1 year ago

In addition to starving the old queue if you have two queues you would also end up getting packets out of order.

I'm not sure what you mean with "retry with a priority" or how you would define that

madninja commented 1 year ago

@disk91 in case you didn't see this

disk91 commented 1 year ago

My main point is to not have fresh new packets to not wait. In a device perspective packets may not be in wrong order for short delay. LNS will ignore packet in wrong order for a single device when long delay as counter will be in the past. Standard device rate is more about an hour than 30s. Order will be wrong from different devices but this don’t care. The proposal of priorty is to always get packet in the fresh queue first, then takes the retry queue. Can be something like:

While (true) {
  while ( priority queue not empty) { 
     PeekOne()
     If ( ! pushIt() ) enqueueInRetryQueue()
  }
  If ( RetryQueue not empty ) {
     PeekOne()
     If ( ! pushIt() ) enqueueInRetryQueue()
  }
 }

If incoming traffic goes to fast we can be blocked in the internal while. We could imagine to have something like on every X frame we exit like replacing it by:

frameProccessed = 0
while ( priority queue not empty && frameProccessed < X ) {
   FrameProcessed++
   PeekOne();
  …
madninja commented 1 year ago

@disk91 my point is that that gets packet out of order right? And the gateway deciding that order is wrong both ways? you're saying it's "don't care" but is that your stance or what all LNS operators already do?

disk91 commented 1 year ago

There is no order in between different devices. the order only matter at a device level. let’s take an exemple:

Original order

Imagine we have B1, then A1, then A2 - no impact at all

imagine B1, A2, A1 the A1 will be dropped but it’s like if it was lost and that’s a normal behavior in lorawan to lost packet.

Also, many chances that the packet has already been delivered by another hotspot.

as in most case, time between A1 and A2 is more about an hour than a minute, we should basically drop A1 instead of relaying an hour later. So timeout on retryqueue also makes sense.

macpie commented 1 year ago

I do not think this would have a lot of impact as the queue (when a hotspot is not connected) is limited to 20 packets and as soon as the hotspot reconnect it will send all of them down the stream.

FYI: Packets are not individually sent, as the hotspot opens a stream (and keeps it open at all time).

Also, if I am not mistaken, old packets will get removed out the queue automatically and the gateway will only keeps the last 20.

disk91 commented 1 year ago

If the queue is limited to 20 entries. I agree, all of this does not have a big impact

madninja commented 1 year ago

If the queue is limited to 20 entries. I agree, all of this does not have a big impact

Indeed it is.. I'll close this as answered for now then