search

ml4ai / tomcat

ToMCAT: Theory of Mind-based Cognitive Architecture for Teams
https://ml4ai.github.io/tomcat/
MIT License
21 stars 7 forks source link

Investigate validity of Aurora data recorded by LSL when Aurora is paused-unpaused #307

Closed CalebUAz closed 2 years ago

CalebUAz commented 2 years ago

For 04-29-2022 testing we decided to pause between tasks and plot for timestamps looks something like this: Untitled Diagram drawio-2

  1. We hit record on labrecorder.
  2. We stop recording on Aurora.
  3. We start the recording again on Aurora.
  4. We stop recording on Aurora.
  5. We start the recording again on Aurora.
  6. We stop recording on Aurora.

When we did this Aurora didn't crash also Labrecorder picks up the fNIRS signals where we last left it.

At LSL index 16613 we stopped aurora recording and next index we resumed recording: image

eduongAZ commented 2 years ago

I am confused looking at the Relative Time from Aurora vs. Timestamp from LSL plot. What I get from the plot is that Relative Time from Aurora was reset back to zero every time Aurora was paused-unpaused. I expect there to be a gap of nothingness in Timestamp from LSL as well, because LSL should not be recording any Aurora data when Aurora was paused.

This plot does not convince me that LSL was correctly recording Aurora data when Aurora is paused-unpaused.

CalebUAz commented 2 years ago

Maybe LSL wasn't implemented that way to take care of "Nothingness". I mean even if LSL places NANs or NULLs in the streams we'd probably remove it during post processing.

Also Aurora is resetting back to zero, because it's creating a new NIRS file and starts streaming from scratch.

eduongAZ commented 2 years ago

Can you read from all NIRx files and plot a data vs. the timestamp? I want to make sure that Aurora does not simply write over old data. When you read and plot from all NIRx files, I expect there to be a few gaps on the plot where there are no data.

CalebUAz commented 2 years ago

Are you talking about NIRx files that is locally stored in aurora? Or What is streamed through LSL?

This plot shows filtered HbO channel 1 vs timestamp from LSL: image

Its not overwriting the existing data collected by LSL in the XDF file, it if was overwriting on existing data then you see an upwards plot from zero not a zig zag plot:

@chinmaib do you concur with what I'm saying?

eduongAZ commented 2 years ago

I expect the "Relative time from Aurora" to keep going upward if we don't pause Aurora. However, all the plot is saying is that the Relative time from Aurora was reset to zero, which could also indicate Aurora is writing from scratch, or it could mean many things, and we don't know what those many things are.

Even when Aurora is paused, I expect the "Relative time from Aurora" to keep going up anyway (not zig zag) because it should know that the experiment is still going and the time should be relative to when LabRecorder first records. It seems that the time is relative to when Aurora was last unpaused, not when LabRecorder first records.

Furthermore, if you look at HbO plot, the data is continuous. There is no indication of pausing. My question is: where did the data come from when Aurora was paused?

CalebUAz commented 2 years ago

We aren't pausing aurora rather we are stopping and starting it again which means its creating a new file on the local machine but resumes streaming onto LSL whenever we hit record.

We stopped at 12:21 after baseline. Started it back at 12:38for Minecraft Saturn A. Stopped it at 12:56. Started it back at 12:59 for Minecraft Saturn B. Finished up with everything at 1:37.

I observed another discrepancy: image The first sheet is fNIRS, 2nd is EEG and 3rd is Pupil capture. All have the same timestamp, which means LSL doesn't stop the time stamp even though one of the streams isn't coming in.

We ran the entire dry run from 11:54AM to 1:37PMwhich is 1hr 42mins (i.e. 6165 sec), fNIRS has a sampling rate of ~10 hz so we should get 61650 samples.

But number of samples in the XDF file for fNIRS is 49,456 which means when we paused the stream there wasn't anything coming in.

chinmaib commented 2 years ago

Looks like Aurora and LSL are behaving just the way we expect. I agree with Caleb, that LSL is picking up streams after we stop and start Aurora automatically. We still need to check if the same behavior happens when Aurora crashes and we restart the entire program.

chinmaib commented 2 years ago

For visualization, the plot of HbO channel 1 vs timestamp from LSL is confusing. Are timestamps from LSL absolute times? The zigzag plot with absolute timestamps from LSL vs relative aurora times is also confusing.

Ideally, a plot of HbO channel 1 vs (absolute) timestamps from LSL should show gaps, but @eduongAZ since there is no data being streamed, nothing is written to the XDF file. Hence we do not see any gaps in the plot 3. But associated timestamps are accurate.

eduongAZ commented 2 years ago

What's the status on this issue? We need to resolve this soon to decide whether to buy new computers, which tend to take a while to be delivered.

From our previous discussion with @kobus-barnard, it seems that we need better plots that should show a straight line when Aurora is paused.

adarshp commented 2 years ago

@CalebUAz - could you please add an explanation for why the issue was closed?

CalebUAz commented 2 years ago

@CalebUAz - could you please add an explanation for why the issue was closed?

@adarshp I was with the impression that we won't be using this approach due to inability to link Aroura's relative time with actual time.

adarshp commented 2 years ago

@CalebUAz - makes sense, thanks!