Absolute timestamps of messages

[andreasdoll]

Hello

I'm trying to get absolute timestamps of messages, something like YYYY-MM-DD hh:mm:ss or POSIX time. Note that I don't know when the device was plugged.

If I understand correctly, the value returned by ics.get_timestamp_for_msg() is the time of message transmit since the device was plugged (in seconds). Because the time of plugin is unknown to me, I thought I can deduce the absolute time from this timestamp and msg.TimeHardware. The relevant snipplet I'm using is:

import ics
import time

for _ in range(3):
   device = ics.find_devices()[0]
   ics.open_device(device)

   tx = ics.SpyMessage()
   tx.Data = (1, 2, 3)
   ics.transmit_messages(device, tx)
   print('TX TimeHardware:', tx.TimeHardware)
   print('TX Timestamp:   ', ics.get_timestamp_for_msg(device, tx))

   messages, _ = ics.get_messages(device)
   rx = messages[0]
   print('RX TimeHardware:', rx.TimeHardware)
   print('RX Timestamp:   ', ics.get_timestamp_for_msg(device, rx))
   print('--')

   ics.close_device(device)
   time.sleep(1)

which outputs:

TX TimeHardware: 0
TX Timestamp:    0.0
RX TimeHardware: 198290384
RX Timestamp:    4.9572595999999995
--
TX TimeHardware: 0
TX Timestamp:    0.0
RX TimeHardware: 256983612
RX Timestamp:    6.424590299999999
--
TX TimeHardware: 0
TX Timestamp:    0.0
RX TimeHardware: 313889723
RX Timestamp:    7.847243075

My questions are: 1) In what unit is msg.TimeHardware reported? 2) Why does TimeHardware not reset when using ics.open_device() (as stated in the documentation)? It does only reset when unplugging the device manually. 3) Is it intended that TX messages return timestamp and TimeHardware as 0? 4) Do I miss the intended method to get absolute timestamps?

Version used: v909.8.

Thanks in advance!

[drebbe-intrepid]

@andreasdoll sorry for the late reply, I will look into this soon (hopefully tomorrow) for you and get back with an answer. Can you let me know what hardware you are using with our API?

[drebbe-intrepid]

@andreasdoll depending on the hardware used there is different behavior here. I'll need that information from you before I can proceed.

[andreasdoll]

Thanks for getting back to me @drebbe-intrepid! The device is a ValueCAN 3, furthermore I use icsneo40.dll (v3.9.8.12) and icsftd3xx.dll (v1.2.0.5)

[drebbe-intrepid]

I'll look into this as soon as I can. This is an older generation of hardware and I believe there was a change in behavior along the ways with newer generations.

[drebbe-intrepid]

@andreasdoll I modified the script to show what ics.get_timestamp_for_msg is doing behind the scenes. Value is in seconds elapsed after 1/1/2007 (we call this ICS epoch time internally).

import ics
import time
from dataclasses import dataclass
import functools

@functools.total_ordering
@dataclass(frozen=True)
class TimeStamp(object):
    id_type: int
    elapsed: int
    is_fixed: bool

    def __eq__(self, other):
        return self.elapsed == other.elapsed

    def __lt__(self, other):
        return self.elapsed < other.elapsed

    def __add__(self, other):
        return self.elapsed + other.elapsed

    def __sub__(self, other):
        return self.elapsed - other.elapsed

    def __str__(self):
        return f"{self.elapsed} (ID: {self.id_type}\tIs Fixed: {self.is_fixed})"

def get_timestamp(msg):
    # bit7 tells us what TYPE of timestamp this is (1=fixed or 0=reference)...
    id_type = msg.TimeStampHardwareID & 0x7F
    elapsed = 0
    if id_type == ics.HARDWARE_TIMESTAMP_ID_NEORED_10NS:
        elapsed = (msg.TimeHardware * ics.NEOVI_RED_TIMESTAMP_1_10NS) + (msg.TimeHardware2 * ics.NEOVI_RED_TIMESTAMP_2_10NS)
    elif id_type == ics.HARDWARE_TIMESTAMP_ID_NEORED_25NS:
        elapsed = (msg.TimeHardware * ics.NEOVI_RED_TIMESTAMP_1_25NS) + (msg.TimeHardware2 * ics.NEOVI_RED_TIMESTAMP_2_25NS)
    elif id_type == ics.HARDWARE_TIMESTAMP_ID_NEORED_10US:
        elapsed = (msg.TimeHardware * ics.NEOVI_RED_TIMESTAMP_1_10US) + (msg.TimeHardware2 * ics.NEOVI_RED_TIMESTAMP_2_10US)
    is_fixed = (msg.TimeStampHardwareID & 0x80) == 0x80
    timestamp = TimeStamp(id_type, elapsed, is_fixed)
    return timestamp

if __name__ == "__main__":
    last_timestamp = 0
    last_hardware = 0
    device = ics.find_devices()[0]
    ics.open_device(device)
    ics.set_rtc(device)
    print(f"Opened {device}...")

    for _ in range(3):
        tx = ics.SpyMessage()
        tx.Data = (1, 2, 3)
        ics.transmit_messages(device, tx)
        print('TX TimeHardware:', tx.TimeHardware)
        print('TX Timestamp:   ', ics.get_timestamp_for_msg(device, tx))

        messages, _ = ics.get_messages(device, False, 1.0)
        rx = messages[0]

        print('RX Timestamp: ', get_timestamp(rx))
        print('RX Timestamp: ', ics.get_timestamp_for_msg(device, rx))
        print('--')

        time.sleep(1)

ValueCAN3

Opened <ics.ics.NeoDevice ValueCAN 3 127444>...
TX TimeHardware: 0
TX Timestamp:    0.0
RX Timestamp:  498064351.00291455 (ID: 9        Is Fixed: False)
RX Timestamp:  498064351.00291455
--
TX TimeHardware: 0
TX Timestamp:    0.0
RX Timestamp:  498064352.0071993 (ID: 9 Is Fixed: False)
RX Timestamp:  498064352.0071993
--
TX TimeHardware: 0
TX Timestamp:    0.0
RX Timestamp:  498064353.0122751 (ID: 9 Is Fixed: False)
RX Timestamp:  498064353.0122751

ValueCAN4-2

Opened <ics.ics.NeoDevice ValueCAN 4-2 V20674>...
TX TimeHardware: 0
TX Timestamp:    0.0
RX Timestamp:  498064226.0009678 (ID: 9 Is Fixed: False)
RX Timestamp:  498064226.0009678
--
TX TimeHardware: 0
TX Timestamp:    0.0
RX Timestamp:  498064227.004492 (ID: 9  Is Fixed: False)
RX Timestamp:  498064227.004492
--
TX TimeHardware: 0
TX Timestamp:    0.0
RX Timestamp:  498064228.0068892 (ID: 9 Is Fixed: False)
RX Timestamp:  498064228.0068892

[drebbe-intrepid]

@andreasdoll Can you point me to the documentation about power cycles resetting the value?

[drebbe-intrepid]

@andreasdoll Also its possible the RTC isn't set correctly (my example sets it). I didn't test it but if you set the RTC in explorer and then run your script you'll probably see different values then starting from 0.

[andreasdoll]

@drebbe-intrepid Thank you, I'll have a look into your example in the coming days.

Regarding the documentation of timestamp resets: here for SpyMessage and here for SpyMessageJ1850 (I didn't use J1850 and hence didn't test whether the documentation applies).

[drebbe-intrepid]

@andreasdoll I'm going to close this out as resolved, if you have anymore issues or questions, feel free to reopen this or open a new issue!

[andreasdoll]

Sorry for the (very) delayed reply @drebbe-intrepid. Basically, I was missing ics.set_rtc(device) and the information about the ICS epoch time. Thanks for clarifying!