Accessing raw EEG signal from Emotiv in Unity

[claudkrog]

Hello,

Is it possible to both Record and Subscribe to the EEG data stream? I am currently recording EEG data, and I have started a session correctly.

However, when I try to subscribe to the data, so that I can GetEEGData, the Unity console says that "A session has not been activated"

My second question is: with GetEEGData, is there already processing on this data, or is this the raw EEG signal?

Additionally, with GetAlphaPower, is EEG preprocessing AND alpha power extraction already applied to this EEG data?

Many thanks, C

[tungntEmotiv]

Hi @claudkrog ,

That's weird. Because to create a record or subscribe EEG data, a session must be created and active beforehand. Could you open comment at SendTextMessage() and WebSocketClient_MessageReceived() function at CortexClient.cs to check the request and response when createSession request called? Send us the log files.

About the second question, Yes, GetEEGData is raw EEG signal . The alpha data is calculated from raw EEG via Emotiv's algorithm :)

[claudkrog]

In the Unity console, it says a Session has been activated successfully (and the recording works well). But later it also says that SubscribeMoreData: A session has not been activated.

This may not be a problem. I have not been able to GetEEGData, so I thought the session activating/not activating may be the problem.

Attached is the log file. Editor_Session_Test_01.log

[tungntEmotiv]

Hi @claudkrog ,

Thank for both your response and question. We found that the current unity-plugin need to update to support createRecord and subscribe data at sametime. We will update soon. While waiting our updating, You can modify the SubscribeMoreData() function at DataStreamManager.cs as below for temporary solution: public void SubscribeMoreData(List<string> streamNameList) { // because the session is activated -> do need check active session // if (_isSessActivated) { foreach(var ele in streamNameList) { _dsProcess.AddStreams(ele); } // Subscribe data _dsProcess.SubscribeData(""); // } // else { // UnityEngine.Debug.Log("SubscribeMoreData: A Session has not been activated."); // } } You don't need to call StartDataStream() , you can SubscribeMoreData() because we have an active session with the headset when a record created . Sorry for inconvenient issue.

[claudkrog]

Thank you very much, I have commented these lines out in order to subscribe data and record at the same time.

I am still not sure how to GetEEGData(), because I am not sure what Channel_t is (an enum it seems, but I cannot find this in the code or documentation).

So I have also tried to GetEEGChannels() in order to learn about their names and the total count of channels, however I have not been able to access this either. In my script I tried:

List mychanlist = new List();

mychanlist = datastrmMgr.GetEEGChannels();

Debug.Log(mychanlist); //This provides a name for a generic list, but does not seem to be accessible more specifically //it seems that it cannot be indexed for a Count because it may be a list of enums

And this (but I do not know what to write for the channel name)

double[] myData = datastrmMgr.GetEEGData(F7); //This is wrong

Would you mind providing some sample code for how to GetEEGData and GetEEGChannels?

Many thanks, C

[tungntEmotiv]

Hi @claudkrog ,

Yes, We will update more usage of these function in the unity-plugin document. But you can reference to our unity-example at https://github.com/Emotiv/cortex-v2-example/tree/master/unity. You can refer DataSubscriber.cs in our example and below sample to get eeg channels and eeg data. if (DataStreamManager.Instance.GetNumberEEGSamples() > 0) { string eegHeaderStr = "EEG Header: "; string eegDataStr = "EEG Data: "; foreach (var ele in DataStreamManager.Instance.GetEEGChannels()) { string chanStr = ChannelStringList.ChannelToString(ele); double[] data = DataStreamManager.Instance.GetEEGData(ele); if (data != null && data.Length > 0) // handle data here } } Let us know if you have any concern. Thanks

[claudkrog]

Thank you. How could I access only the data from the F4 electrode, for example?

Currently, when I debug DataStreamManager.Instance.GetNumberEEGSamples() it is always 0. I do not see any errors in the log. The recording is working, however.

[tungntEmotiv]

@claudkrog To get data for F4 you can do DataStreamManager.Instance.GetEEGData(Channel_t.CHAN_F4); Could you add log to OnStreamDataReceived function at DataStreamProcess.cs private void OnStreamDataReceived(object sender, StreamDataEventArgs e) { if (e.StreamName == DataStreamName.EEG) { // Add log to make sure you receive eeg data UnityEngine.Log(" receive eeg data " + e.Data.Count); EEGDataReceived(this, e.Data); } In addition, in our example, we will add the subscribed data to buffered and retrieve the data at GetDataPoker() at DataProcessing.cs, you can reference the example . You also can customize your code to handle EEGDataReceived signal yourself and do not need add to buffer. It might be simpler for your project.

[claudkrog]

When I add this log to the OnStreamDataReceived function, I do not see the debug.log message in Unity (for any of the data: eeg, facial expressions, etc).

Yes, I think getting EEGDataReceived would be better for my project, since it will already be handled by the buffer. I have not been able to access EEGDataReceived in my scripts however. I appreciate the help!

[tungntEmotiv]

Hi @claudkrog , In normal case, you will receive OnStreamDataReceived after you call subscribe a data stream. To subscribe EEG data, please make sure you have authorized your App with a PRO license. Please reference to https://github.com/Emotiv/cortex-v2-example/blob/master/unity/Assets/Plugins/ConnectToCortex.cs. If the issue still happen, Please uncomment some lines at WebSocketClient_MessageReceived() and SendTextMessage() at CortexClient.cs to check all requests and responses. Then provide us the log. Thanks

[claudkrog]

We should have the Pro license. Here is the log file: Editor_aug20.log

Is there also a way to GetEEGAlphaPower(Channel_t.CHAN_F4) which has already finished with the buffer, similar to EEGDataReceived?

If possible, could you tell me the EEGDataReceived and GetEEGAlphaPower(Channel_t.CHAN_F4) data structure/type? Thank you so much for the clarification.

Many thanks, C

[tungntEmotiv]

Hi @claudkrog ,

The log file shown that "MessageErrorRecieved :code -32004 message The headset is currently not available.method name createSession" -> might wrong headsetId when create session. Because i don't know how to customize the unity-plugin , So i will list the order of functions as below:

1. DataStreamManager.Instance.SetAppConfig(clientId, clientSecret, ) -> to set credential information for your App
2. DataStreamManager.Instance.StartAuthorize(yourLicenseId) -> to authorize your App with a license , the function also check the license valid or not
3. DataStreamManager.Instance.StartDataStream(List streamNameList, string headsetId) -> connect to the headset with headsetId, then create a working session with the headset , then subscribe streams in the streamList
4. You can start and stop Record or subscribe more data

For example: `DataStreamManager.Instance.SetAppConfig("appclientId", "appclientSecret"); DataStreamManager.Instance.StartAuthorize("yourlicenseID); // wait event DataStreamManager.LicenseValidTo -> inform that your license is valid

// subscribe dev and eeg List dataStreamList = new List(){"dev", "eeg"}; DataStreamManager.Instance.StartDataStream(dataStreamList, "theheadsetID"); // the headsetID For Eg: "INSIGHT-C4826128"

// wait response at DataStreamManager::OnSubscribedOK -> check which data are subscribed successfully // Then the event of data will be handled at DataStreamProcess::OnStreamDataReceived

//Now You can start a record or subscribe more data RecordManager.Instance.StartRecord() or DataStreamManager.Instance.SubscribeMoreData()`

Do you mean DataStreamManager.Instance.GetEEGData(Channel_t.CHAN_F4) ? -> it get from buffer. The eeg data will add to 2 dimension buffer with column is channel. So GetEEGData(Channel_t.CHAN_F4) -> will return the double array of F4 data. You can reference to GetDataPoker() in DataProcessing.cs in the unity example.

But the event DataStreamProcess. EEGDataReceived () -> data before adding to buffer. The data is ArrayList of double type and has channels name as order ["TIMESTAMP","COUNTER","INTERPOLATED", "AF3","T7","Pz","T8","AF4", "RAW_CQ", "MARKER_HARDWARE"] . Cortex send the data with sample rate 128 or 256 Hz depend on you settings of headset.

Next time you send me the log, please uncomment below line at SendTextMessage() in CortexClient.cs UnityEngine.Debug.Log(request.ToString());

Thanks

[claudkrog]

I think it may say there is a wrong headset because I did change headsets. I updated the headset ID for the second headset, so I don't know why I see this message. The recording works, so I assumed the second headset ID was correctly input.

In order to both record and subscribe to eeg data, I have this code :

DataStreamManager.Instance.SetAppConfig(clientID, secret);

DataStreamManager.Instance.StartAuthorize(my license ID); //I just added the license ID, it seemed to work without any parameters previously

SessionHandler.Instance.Create(Authorizer.Instance.CortexToken, headsetID, true);

dataStreamList = new List() { DataStreamName.EEG };

DataStreamManager.Instance.StartDataStream(dataStreamList, headsetID); //I was told this was not needed, because SubscribeData is called next, but I include it anyways

DataStreamManager.Instance.SubscribeMoreData(new List() { DataStreamName.EEG});

RecordManager.Instance.StartRecord("BCI", subjectID);

I have wait events set up, and record works great.

Looking at our account, I see a new message now (attached picture). Do I need to tick this box now? Thank you! C

editor-aug21.txt

[tungntEmotiv]

Hi @claudkrog ,

Because you have subscribe EEG data at dataStreamList = new List() { DataStreamName.EEG }; DataStreamManager.Instance.StartDataStream(dataStreamList, headsetID); So you do not need DataStreamManager.Instance.SubscribeMoreData(new List() { DataStreamName.EEG}); -> only if you subscribe other data streams.

Yes, you need to tick the box to your App has access to EEG data.

[claudkrog]

My app actually does have EEG access. Unfortunately, I am still not receiving any messages in Unity from this code

private void OnStreamDataReceived(object sender, StreamDataEventArgs e) { if (e.StreamName == DataStreamName.EEG) { // Add log to make sure you receive eeg data UnityEngine.Log(" receive eeg data " + e.Data.Count); EEGDataReceived(this, e.Data); }

I have never seen "receive eeg data". I still don't think I am receiving data.

[tungntEmotiv]

Hi @claudkrog ,

1) In the attached log, i see that the problem is a session is created before you call DataStreamManager.Instance.StartDataStream() .It leads to issue Session is activated but for headset EPOCX-E20205DA So you should remove SessionHandler.Instance.Create(Authorizer.Instance.CortexToken, headsetID, true); in your code. Because the function DataStreamManager.Instance.StartDataStream() will create session with a wanted headset so you do not need the step. Please check my above guideline.

2) About DataStreamManager.Instance.StartAuthorize(my license ID); //I just added the license ID, it seemed to work without any parameters previously, Yes you can set empty string for license here because Cortex will help to choose a valid license in your license list.

From beginning, the unity-plugin aims to demo how to use Cortex API, so it might not be easy enough for developer integrate to their apps. We are really sorry about that. However, We are willing to hear developer's feedback to improve the unity-plugin. So if you have any feedback or expectation about the plugin please let us know. Thank

[claudkrog]

When I remove this line of code SessionHandler.Instance.Create(Authorizer.Instance.CortexToken, headsetID, true);

I am now not recording EEG data, and still not getting EEG data.

It seems the SessionHandler.Instance.Create(Authorizer.Instance.CortexToken, headsetID, true); is at least needed to record EEG data

It seems that none of these Debugs are ever printed (DataStreamProcess.cs) :

private void OnStreamDataReceived(object sender, StreamDataEventArgs e) { Debug.Log("EEG Data has been received!");

        if (e.StreamName == DataStreamName.EEG)
        {
            EEGDataReceived(this, e.Data);
            Debug.Log("got the eeg data " + e.Data.Count);
        }
        else if (e.StreamName == DataStreamName.Motion)
        {
            MotionDataReceived(this, e.Data);
            Debug.Log("star  motion");
        }
        else if (e.StreamName == DataStreamName.PerformanceMetrics)
        {
            PerfDataReceived(this, e.Data);
            Debug.Log("star performance");
        }
        else if (e.StreamName == DataStreamName.BandPower)
        {
            BandPowerDataReceived(this, e.Data);
            Debug.Log("star  band power");
        } 
        else if (e.StreamName == DataStreamName.DevInfos)
        {
            DevDataReceived(this, e.Data);
            Debug.Log("star  devInfos");
        } 
        else if (e.StreamName == DataStreamName.FacialExpressions) 
        {

            FacialExpReceived(this, e.Data);
            Debug.Log("star facials");
        }
        else if (e.StreamName == DataStreamName.MentalCommands) 
        {

            MentalCommandReceived(this, e.Data);
            Debug.Log("star  mental commands");
        }
        else if (e.StreamName == DataStreamName.SysEvents)
        {
            SysEventsReceived(this, e.Data);
            Debug.Log("star  sysevents");
        }
    }

[tungntEmotiv]

Hi @claudkrog , The above finding based on your log file. If you followed up my guideline but the issue still happen. Could you send me a part of your project which contain data subscribing and record creating? i will try to debug on your project. My email tungnguyen@emotiv.com.

[claudkrog]

I have realized the Debug Log messages from OnStreamDataReceived(object sender, StreamDataEventArgs e) in DataStreamProcess.cs may not show up even if the streaming is working.

However, I think the problem is that when I print the EEG data or the Alpha data, I always get "0". I will send my project file. Thank you for your time. C

[tungntEmotiv]

Hi @claudkrog ,

Please check my email. Thanks

[claudkrog]

Could you tell me if the data provided from GetAlphaData(Channel_t.CHAN_F4) (for example)

is already naturally logged?

I need to know if this alpha power data is already the Ln(Chan F4 alpha power).

I am doing a further analysis where I need to take the natural log of F4, but if it is already done in Emotiv's algorithm, I do not want to do this a second time.

Thank you, C

[tungntEmotiv]

Hi @claudkrog

is already naturally logged? I need to know if this alpha power data is already the Ln(Chan F4 alpha power). No, it haven't applied naturally log

[claudkrog]

Although I can record, and Get EEG Data / Alpha power at the same time now, I seem unable to inject markers into my recording while I am getting EEG data. Is there any modification of the code I could try?

Currently, When I inject markers into the recording I receive this error: IndexOutOfRangeException (line of code = double f4EEGData = f4Data[0];, which is within:

if (_dataStreamMgr.GetNumberEEGSamples() > 0) { double[] f4Data = _dataStreamMgr.GetEEGData(Channel_t.CHAN_F4); if (f4Data != null ) { double f4EEGData = f4Data[0]; } }

The marker is correctly injected into the recording, and then I receive the error, and no longer get eeg data.

If I do not call my inject markers function, I do not have any errors, except that I need to inject markers. Thank you

[tungntEmotiv]

Hi @claudkrog ,

We used to fix a crash related to markers at EEG stream at AddDataToBuffer() function of EegMotionDataBuffer.cs in PR https://github.com/Emotiv/unity-plugin/pull/6/files . Did your code contain this fix ?

When you inject marker , a marker object will be included in EEG data stream from Cortex. In that PR, we will exclude marker data in both data buffer and channel list

[claudkrog]

My code currently contains:

public override void AddDataToBuffer(ArrayList data) { int nChannels; if (_dataType == DataType.EEG) nChannels = data.Count -1; //exclude MARKERS channel else nChannels = data.Count;

for (int i=0 ; i < nChannels; i++) {
        if (data[i] != null) {
             double eegData = Convert.ToDouble(data[i]);
              bufHi[i].AppendData(eegData);
            }
        }

      }

Should I modify this code? Many thanks

[tungntEmotiv]

Hi @claudkrog ,

Yes, you should modify the function ` public override void AddDataToBuffer(ArrayList data) { if (data.Count > _channels.Count) { UnityEngine.Debug.Log("AddDataToBuffer: data contain markers channels."); }

        for (int i=0 ; i <  _channels.Count; i++) {
            if (data[i] != null) {
                double eegData = Convert.ToDouble(data[i]);
                bufHi[i].AppendData(eegData);
            }
        }    
    }

` Thanks

[claudkrog]

I seem to have the same issue, with the code fix

[tungntEmotiv]

The root cause of this issue is json object parsing at CortexClient.cs line 258 when EEG data contain a Marker object. After applied below change, the issue has fixed. So i will close this ticket if (ele.Type == JTokenType.Array){ foreach (var item in ele){ if (item.Type == JTokenType.Object) { // Ignore marker data UnityEngine.Debug.Log("marker object " + item); } else data.Add(Convert.ToDouble(item)); } }

We will update the unity-plugin fix soon. Thanks