search

OceanDataTools / openrvdas

An open source data acquisition system designed for use on research vessels and other scientific installations
http://openrvdas.org
Other
39 stars 20 forks source link

Feature request: new parameter to tell CDS to bundle records #376

Closed davidpablocohn closed 3 months ago

davidpablocohn commented 4 months ago

Parameter should tell CachedDataServer to only return records every N seconds, and to return them as a bundle.

Requested by @ellapiet.

ellapiet commented 4 months ago

@davidpablocohn I had a play with adding in that single line of asyncio.sleep() in the cached data reader. Looks like it isn't going to give us the functionality we were hoping for as records are still returned singularly rather than in a list

davidpablocohn commented 4 months ago

Hmmmm - okay, thanks for poking at that. I'll have a look this week and see if I'm able to kludge things in a not-too-kludgey way.

On Sun, Apr 28, 2024 at 6:47 PM ellapiet @.***> wrote:

@davidpablocohn https://github.com/davidpablocohn I had a play with adding in that single line of asyncio.sleep() in the cached data reader. Looks like it isn't going to give us the functionality we were hoping for as records are still returned singularly rather than in a list

— Reply to this email directly, view it on GitHub https://github.com/OceanDataTools/openrvdas/issues/376#issuecomment-2081766498, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFO7V3S2I4IR6LAAC3LFKK3Y7WRCLAVCNFSM6AAAAABG5IAJEOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBRG43DMNBZHA . You are receiving this because you were mentioned.Message ID: @.***>

davidpablocohn commented 4 months ago

@ellapiet - is there a case where you'd like to have different time window bundles for different variables (e.g. bundle heading every 10 seconds, but bundle temps every 30 seconds)? Or would it be adequate to set a single window for all the variables, e.g. "Give me all the DASRecords that I've subscribed to that have arrived in the last N seconds"?

If the latter, I think I can do a clean and easy implementation inside CachedDataReader.

davidpablocohn commented 4 months ago

I think this change in branch issue_376 does what's needed: https://github.com/OceanDataTools/openrvdas/blob/issue_376/logger/readers/cached_data_reader.py

One further improvement would be to base the bundle window on DASRecord timestamps vs system time. Do you have feelings on which way that should go?

On Sun, Apr 28, 2024 at 7:18 PM David Pablo Cohn @.***> wrote:

Hmmmm - okay, thanks for poking at that. I'll have a look this week and see if I'm able to kludge things in a not-too-kludgey way.

On Sun, Apr 28, 2024 at 6:47 PM ellapiet @.***> wrote:

@davidpablocohn https://github.com/davidpablocohn I had a play with adding in that single line of asyncio.sleep() in the cached data reader. Looks like it isn't going to give us the functionality we were hoping for as records are still returned singularly rather than in a list

— Reply to this email directly, view it on GitHub https://github.com/OceanDataTools/openrvdas/issues/376#issuecomment-2081766498, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFO7V3S2I4IR6LAAC3LFKK3Y7WRCLAVCNFSM6AAAAABG5IAJEOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBRG43DMNBZHA . You are receiving this because you were mentioned.Message ID: @.***>

ellapiet commented 3 months ago

Sorry @davidpablocohn I'm out at sea right now and missed this one. Changes look great!