IMU framerate and timestamping issues

[ozgunkaratas]

Hello, as mentioned here #60 , i have a similar problem regarding multiple sensor usage. i modified the streamrecorder app to also have 3 separate threads to obtain IMU_MAG, IMU_ACCEL and IMU_GYRO sensor information and these are written to .txt files and bundled in a .tar file at the end just like how the PV camera data is bundled in tar. If I want to record the data by only using the IMUs, the framerate suffers from fluctuations, albeit not too much. however, if i want to access PV and IMU at the same time, the fluctuations are so much that HL IMU data does not match what i am doing in the real world at one instance in time.

1) When using the SensorVisualizationApp and printing the IMU data to the debug screen, the accelerometer publishes at about 12Hz and Gyroscope at about 20Hz. These seems to be not affected by fluctuations.

2) Obtaining IMU sensor information and recording it in Hololens storage leads to moderate fluctuations in data output rate when only the IMU is accessed. If RM cameras or the PV camera are accessed, the fluctuations are severe.

3) I am using the same method that is used for PV to obtain the timestamp for the IMU and i extended it to include other time variables as well. this extension can be found below (in this particular case, for the accelerometer):

     //time related data is created here
    float                     gettime                        = m_timer.GetTime();
    static unsigned long long getsystemrelativetime          = m_timer.GetSystemRelativeTime();
    static double             getsystemrelativetimeinseconds = m_timer.GetSystemRelativeTimeInSeconds();
    static unsigned long long getfiletime                    = m_timer.GetFileTime();

    //timerconverter related data
    HundredsOfNanoseconds timestamp             = m_converter.RelativeTicksToAbsoluteTicks(HundredsOfNanoseconds((long long)m_prevTimestamp));
    HundredsOfNanoseconds universalstamp        = m_converter.UniversalToUnixTime(fileTime);
    HundredsOfNanoseconds filetimetoabsticks    = m_converter.FileTimeToAbsoluteTicks(fileTime);
    HundredsOfNanoseconds reltoabsticksoffset   = m_converter.CalculateRelativeToAbsoluteTicksOffset();

    sprintf_s(printString, "####Accel: % 3.6f % 3.6f % 3.6f %f %I64d %I64d %I64d %f %llu %f %llu %I64d %I64d\n",
        sample.x,
        sample.y,
        sample.z,
        sqrt(sample.x * sample.x + sample.y * sample.y + sample.z * sample.z),
        (lastSocTickDelta * 1000) / timeStamp.HostTicksPerSecond,
        timestamp.count(),
        universalstamp.count(),
        gettime,
        getsystemrelativetime,
        getsystemrelativetimeinseconds,
        getfiletime,
        filetimetoabsticks.count(),
        reltoabsticksoffset.count()
    );

here i am using the Timer.h and the TimeConverter.h headers to obtain time related values.

4) Timestamping is directly affected from the sensor outputrate in the sense that the timestamps of the IMU drift into the future whereas the timestamps of the PV camera reflects the correct time in the real world.

With that said, I would like to know if it is possible to obtain correct timestamp information for each particular IMU frame or will it be always affected by the sensor publish rate fluctutations? This problem makes it hard to compare HL IMU with high grade imus because of the drift in timestamps and sever fluctuations in refresh rates.

thank you in advance,

[ozgunkaratas]

I would like to update my question as I have found out that the app still records data even the stop button is pressed for some indefinite time. I have come to this conclusion because after I compare the IMU data with my correctly timestamped high-grade IMU data, their movements match up. The camera quits recording data after I press the stop button and I can verify this with the last timestamp that I obtained from the images. To work around this issue, I am cutting the IMU information from my recorded data after the camera turns off properly.

Additionally, I have found out that the IMU data drifts significantly as time goes on, have a look at the attached matlab figure for visualization where I compare the data with the high grade XSENS IMU (in this particular case z-axis of the gyroscope) I have marked the peaks so that you can see the increasing time drift of the IMU from the beginning of the capture till the end.

Is this unavoidable? The average accelerometer rate becomes 10Hz and the average gyroscope rate becomes 13Hz with this case, albeit the time difference between consequtive data points is fluctuating severely, have a look at the following images.

I would appreciate some insight on this matter,

thank you in advance.

[ozgunkaratas]

Hello everybody,

i would like to close the issue, since i found out that this delayed "behavior" was because of incorrect plotting against the samples. if we plot against time, the movement patterns are synching up nicely. on the other hand, irregular sampling rates can be worked around with the implementation of interpolation algorithms.