timestamp miscalculation for LF -RF grayscale cameras

[jondebug]

I have downloaded the grayscale streams but when trying to display them in the order their timestamp describes i find about 20% of the images received a wrong timestamp. I can tell this is the case since when for 10 sequentially timestamped images all are stamped in chronological order but one of them is clearly from the future (its given a timestamp smaller then its actual timestamp) .

Is this a known problem or a problem specifically with my unit? Is there a solution for this?

Thanks for the help, Jonathan

[kianwei96]

@jondebug , I faced the same issue as you with the mis-stamped LF-RF images in StreamRecorder. Have not figured out the reason for it, but I doubt it's a device specific. My workaround is to stream the data and save it on a workstation, but it would be awesome if StreamRecorder could be fixed for this..

[jondebug]

Hi Kian thanks for replying!

Could you explain what you mean by streaming the data to a workstation?

I assume you're not using StreamRecorder at all and somehow saving the streams directly on to an external device?

If this is the case, does it solve the problem?

Thanks for the help,

Jonathan

On Thu, Dec 30, 2021, 08:50 Kian Wei Ng @.***> wrote:

@jondebug https://github.com/jondebug , I faced the same issue as you with the mis-stamped LF-RF images in StreamRecorder. Have not figured out the reason for it, but I doubt it's a device specific. My workaround is to stream the data and save it on a workstation, but it would be awesome if StreamRecorder could be fixed for this..

— Reply to this email directly, view it on GitHub https://github.com/microsoft/HoloLens2ForCV/issues/117#issuecomment-1002896004, or unsubscribe https://github.com/notifications/unsubscribe-auth/AN2Y6MFLA3QIVV3MXVG25QDUTP6MPANCNFSM5JTH4FGA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

[kianwei96]

Hi Jonathan,

There are several repos mentioned in other repos that provide a Unity based solution to stream research mode data from the HL2 device to a laptop/desktop wirelessly. The streamed data seems to not encounter this "jerkiness" issue, as far as I can tell! But you'll have to configure some of the code to stream and save the data that you want.

Alternatively, you could try and see if you can roll back to an older OS version on Hololens. I'm only seeing this timestamp miscalculation on a device with a newer OS. Worked with the same code on a device with an older OS installed. Never investigated it further, but it could be something that you'd like to look into!

Thanks, Kian Wei

[revkusz]

Hello!

I noticed this issue too. After i tweaked the code a little i noticed that this error appears when the CPU is really busy. The weird thing is that the ResearchModeApi gives the wrong frames for the wrong time stamps so i noticed that some frames comes from the API 3-4 frames later. My solution to this problem was that i added some wait for the CameraWriteThread in RMCameraReader.cpp when there is no frame to write bc without that the threads endlessly try to check if there is a new frame to write. This quick fix saved a lot of CPU cycles and poof the problem was disappeared for me. Maybe its fixing #86 too.

Richárd

[KellyYe]

@revkusz Hi Richárd,

Brilliant idea! I also think it a CPU problem, but failed to solve the problem by adding wait in CameraWriteThread. I added unique_lock and check if no new frame comes then wait. The program stuck or made no difference and the problem still there. How did you add it? Thanks for your help!

void RMCameraReader::CameraWriteThread(RMCameraReader* pReader)
{
    while (!pReader->m_fExit)
    {
        std::unique_lock<std::mutex> storage_lock(pReader->m_storageMutex);
        if (pReader->m_storageFolder == nullptr)
        {
            pReader->m_storageCondVar.wait(storage_lock);
        }

        std::unique_lock<std::mutex> newFrame_lock(pReader->m_newFrameMutex);
        if (!(pReader->m_pSensorFrame) ||
            (pReader->m_pSensorFrame) && !(pReader->IsNewTimestamp(pReader->m_pSensorFrame)))
        {
            //if (!(pReader->IsNewTimestamp(pReader->m_pSensorFrame)))
            {
                pReader->m_newFrameCondVar.wait(newFrame_lock);
            }
        }

        std::lock_guard<std::mutex> reader_guard(pReader->m_sensorFrameMutex);
        if (pReader->m_pSensorFrame)
        {
            if (pReader->IsNewTimestamp(pReader->m_pSensorFrame))
            {
                pReader->SaveFrame(pReader->m_pSensorFrame);
            }
        }
    }   
}

Kelly

Hello!

I noticed this issue too. After i tweaked the code a little i noticed that this error appears when the CPU is really busy. The weird thing is that the ResearchModeApi gives the wrong frames for the wrong time stamps so i noticed that some frames comes from the API 3-4 frames later. My solution to this problem was that i added some wait for the CameraWriteThread in RMCameraReader.cpp when there is no frame to write bc without that the threads endlessly try to check if there is a new frame to write. This quick fix saved a lot of CPU cycles and poof the problem was disappeared for me. Maybe its fixing #86 too.

Richárd