iternio / ovms-link

14 stars 6 forks source link

Collect Data at High Frequency, Send Data in Batches #23

Open kezarjg opened 2 months ago

kezarjg commented 2 months ago

I would like to propose an enhancement to support collecting data at a high frequency and sending it in batches. Specifically, I am interested in understanding if there is an API call to support:

  1. Collect consumption data at high frequency – For example, gathering data every 5 seconds.
  2. Send data in batches – Instead of sending each data point individually, it would be sent in a batch after a specified interval or when certain conditions (e.g., network availability) are met.

This would be particularly useful in scenarios where:

jason-abrp commented 2 months ago

We actually have a bulk endpoint that's available for exactly this sort of thing. I'll update the API docs to include it, but this is definitely possible.

jason-abrp commented 2 months ago

https://documenter.getpostman.com/view/7396339/SWTK5a8w#4f77e0b8-aa60-4a6e-83df-3c3f7fe19aec

The format is fairly simple, and the innermost "tlm_list" elements are identical to the tlm field of the current format.

kezarjg commented 2 months ago

OK. I'll take a look. Do the entries need to be sorted by time? I've been getting some odd jumps in my trip data (like when I might have connectivity problems)

image

jason-abrp commented 2 months ago

Assuming the timestamp (UTC seconds) on the entry itself is correct, it should be added to the database and displayed properly. We do discard out of order data for calibration though, as we need to go sequentially.

It is definitely better to send the points in sorted by time though, oldest to newest.

kezarjg commented 2 months ago

https://documenter.getpostman.com/view/7396339/SWTK5a8w#4f77e0b8-aa60-4a6e-83df-3c3f7fe19aec

The format is fairly simple, and the innermost "tlm_list" elements are identical to the tlm field of the current format.

Does the API key need to be authorized for bulk uploads? I'm receiving the following error; 401 Unauthorized Key

kezarjg commented 2 months ago

@jason-abrp My current thinking is to send bulk telemetry every 60 seconds. Will that cause any issues with ABRP?

jason-abrp commented 2 months ago

That should be fine, but maybe a little choppy on the displayed SoC in-app while driving since it could then lag behind by up to a minute.

kezarjg commented 2 months ago

OK. I'll do some testing with every 10 seconds.

kezarjg commented 1 month ago

@jason-abrp - Any suggestion what to do with data when no connection is available. Is there value to ABRP to send the data 'later'? Does it help with average consumption or charging data for vehicles?

jason-abrp commented 1 month ago

Sending the data later is good for filling in missing segments in your drive history, but it's not really used by the calibration process. Up to you if you want to include that.