This time state is automatically maintained by our thread-local clocks, and automatically injected into the logged data by the RecordingStream's ingestion pipeline.
These clocks even keep track of our builtin timelines (log_time, log_tick) without any user intervention.
This allows users to fully benefit from our multi-timeline model with minimal effort. Nice.
But, sometimes you want to partially or even fully opt-out from this automatic time tracking.
An example of fully opting out is when ingesting an existing dataset from another database: the data may be dating from years prior and adding a log_time to it doesn't make sense.
You want a carbon copy of that original data, no more no less, and for that you need to opt-out of automatic time-tracking and have a way of specifying extra time data.
An example of partially opting out is when you want to associate some extra time data with one single specific log call, in addition to the automatic time.
In that case it doesn't make sense to set some global time state just to unset it immediately after the log call.
A common example of that is scalars: they use timelines as their X axis. You want to be able to easily custom their X axis and still have them to exist in all other timelines.
Finally, today, we already expose a timeless flag on all our logging APIs.
This is effectively a full opt-out mechanism: timeless=true will drop all automatic time metadata when logging.
We can use this as a basis for our proposed stateless time APIs.
Proposal
Keep the timeless flag as-is.
Add an optional TimePoint parameter to our logging APIs.
Apply the correct behavior depending on both the timeless & timepoint parameters
The final time metadata for the logged data will always be the union of the time specified in the timepoint parameter and the time from the stateful clocks.
In case of conflicts, the values from timepoint overwrites those from the stateful clocks.
For a partial opt-out behavior, you only need to pass in a timepoint parameter.
For a full opt-out behavior, you set timeless=trueand pass in a timepoint parameter.
Context
Our 3 SDKs now support our stateful time APIs (although there are still some rough edges), e.g.:
This time state is automatically maintained by our thread-local clocks, and automatically injected into the logged data by the
RecordingStream
's ingestion pipeline. These clocks even keep track of our builtin timelines (log_time
,log_tick
) without any user intervention. This allows users to fully benefit from our multi-timeline model with minimal effort. Nice.But, sometimes you want to partially or even fully opt-out from this automatic time tracking.
An example of fully opting out is when ingesting an existing dataset from another database: the data may be dating from years prior and adding a
log_time
to it doesn't make sense. You want a carbon copy of that original data, no more no less, and for that you need to opt-out of automatic time-tracking and have a way of specifying extra time data.An example of partially opting out is when you want to associate some extra time data with one single specific log call, in addition to the automatic time. In that case it doesn't make sense to set some global time state just to unset it immediately after the log call. A common example of that is scalars: they use timelines as their X axis. You want to be able to easily custom their X axis and still have them to exist in all other timelines.
Finally, today, we already expose a
timeless
flag on all our logging APIs. This is effectively a full opt-out mechanism:timeless=true
will drop all automatic time metadata when logging. We can use this as a basis for our proposed stateless time APIs.Proposal
Keep the
timeless
flag as-is.Add an optional
TimePoint
parameter to our logging APIs.E.g. for python:
Of course that means we'll also have to expose new APIs to make it possible and user-friendly to build up TimePoints in python.
Rust:
And something similar for C++.
Apply the correct behavior depending on both the
timeless
&timepoint
parametersThe final time metadata for the logged data will always be the union of the time specified in the
timepoint
parameter and the time from the stateful clocks. In case of conflicts, the values fromtimepoint
overwrites those from the stateful clocks.For a partial opt-out behavior, you only need to pass in a
timepoint
parameter. For a full opt-out behavior, you settimeless=true
and pass in atimepoint
parameter.