Emotiv / unity-plugin

Unity support for Cortex V3
MIT License
35 stars 12 forks source link

Need help in accessing mental command from a profile #2

Closed NarinderBrar closed 4 years ago

NarinderBrar commented 4 years ago

We are unable to access mental commands from a trained profile.

Following function from DataStreamManager class printing only NEUTRAL command with 0 pow value.

    private void OnMentalCommandReceived(object sender, ArrayList data)
    {
        if (_mentalCommandLists == null || _mentalCommandLists.Count != data.Count)
        {
            UnityEngine.Debug.LogAssertion("OnMentalCommandReceived: Mismatch between data and label");
            return;
        }

        UnityEngine.Debug.Log("data count: " + data.Count);

        double time     = Convert.ToDouble(data[0]);
        string act      = Convert.ToString(data[1]);
        double pow      = Convert.ToDouble(data[2]);

        MentalCommandEventArgs comEvent = new MentalCommandEventArgs(time, act, pow);

        string comListsStr = "";

        foreach (var ele in _mentalCommandLists)
        {
            comListsStr += ele + " , ";
        }

        UnityEngine.Debug.Log("MentalCommand labels: " +comListsStr);
        UnityEngine.Debug.Log("MentalCommand datas : " +comEvent.Time.ToString() + " , " + comEvent.Act+ " , " + comEvent.Pow);
    }

Also with the help of BCITraining class, we are able to create a profile, and can also print the list of all available profiles, even from native emotive applications.

But we are not able to apply these trained commands in applications.

Thanks for help in advance.

NarinderBrar commented 4 years ago

Update, According to following link in "Getting live actions" https://emotiv.gitbook.io/cortex-api/bci

We have query profile, and got 2 existing profiles.

When we try to load one profile, got following 2 msgs in console.

Msg 1: Received: Invalid Parameters. UnityEngine.Debug:Log(Object) EmotivUnityPlugin.CortexClient:WebSocketClient_MessageReceived(Object, MessageReceivedEventArgs) (at Assets/Plugins/Emotiv-Unity-Plugin/Src/CortexClient.cs:234) WebSocket4Net.WebSocket:FireMessageReceived(String) WebSocket4Net.Command.Text:ExecuteCommand(WebSocket, WebSocketCommandInfo) WebSocket4Net.WebSocket:ExecuteCommand(WebSocketCommandInfo) WebSocket4Net.WebSocket:OnDataReceived(Byte[], Int32, Int32) WebSocket4Net.WebSocket:client_DataReceived(Object, DataEventArgs) SuperSocket.ClientEngine.ClientSession:OnDataReceived(Byte[], Int32, Int32) SuperSocket.ClientEngine.AuthenticatedStreamTcpSession:OnDataRead(IAsyncResult) System.Runtime.CompilerServices.AsyncTaskMethodBuilder1:SetResult(Int32) Mono.Net.Security.<StartOperation>d__58:MoveNext() System.Runtime.CompilerServices.AsyncTaskMethodBuilder1:SetResult(AsyncProtocolResult) Mono.Net.Security.d23:MoveNext() System.Runtime.CompilerServices.AsyncTaskMethodBuilder:SetResult() Mono.Net.Security.d24:MoveNext() System.Runtime.CompilerServices.AsyncTaskMethodBuilder1:SetResult(Nullable1) Mono.Net.Security.d25:MoveNext() System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetResult(Int32) Mono.Net.Security.d66:MoveNext() System.Threading._ThreadPoolWaitCallback:PerformWaitCallback()

Msg 2: MessageErrorRecieved :code -32602 message Invalid Parameters.method name setupProfile UnityEngine.Debug:Log(Object) EmotivUnityPlugin.DataStreamProcess:MessageErrorRecieved(Object, ErrorMsgEventArgs) (at Assets/Plugins/Emotiv-Unity-Plugin/Src/DataStreamProcess.cs:297) EmotivUnityPlugin.CortexClient:WebSocketClient_MessageReceived(Object, MessageReceivedEventArgs) (at Assets/Plugins/Emotiv-Unity-Plugin/Src/CortexClient.cs:236) WebSocket4Net.WebSocket:FireMessageReceived(String) WebSocket4Net.Command.Text:ExecuteCommand(WebSocket, WebSocketCommandInfo) WebSocket4Net.WebSocket:ExecuteCommand(WebSocketCommandInfo) WebSocket4Net.WebSocket:OnDataReceived(Byte[], Int32, Int32) WebSocket4Net.WebSocket:client_DataReceived(Object, DataEventArgs) SuperSocket.ClientEngine.ClientSession:OnDataReceived(Byte[], Int32, Int32) SuperSocket.ClientEngine.AuthenticatedStreamTcpSession:OnDataRead(IAsyncResult) System.Runtime.CompilerServices.AsyncTaskMethodBuilder1:SetResult(Int32) Mono.Net.Security.<StartOperation>d__58:MoveNext() System.Runtime.CompilerServices.AsyncTaskMethodBuilder1:SetResult(AsyncProtocolResult) Mono.Net.Security.d23:MoveNext() System.Runtime.CompilerServices.AsyncTaskMethodBuilder:SetResult() Mono.Net.Security.d24:MoveNext() System.Runtime.CompilerServices.AsyncTaskMethodBuilder1:SetResult(Nullable1) Mono.Net.Security.d25:MoveNext() System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetResult(Int32) Mono.Net.Security.d66:MoveNext() System.Threading._ThreadPoolWaitCallback:PerformWaitCallback()

tungntEmotiv commented 4 years ago

Hi @NarinderBrar 1) For first question:

Following function from DataStreamManager class printing only NEUTRAL command with 0 pow value.

It is default data because no profile. You need a trained profile (a least one action trained) and subscribe "com", you may see different value. It look like Live mode when you do with BCI Application. We have some minor update. Please pull the latest. Currently, we have not supported Training with a UI yet. So i will guide you with some snippets: ` // 1.change app config and start authorize such as unity-example ...

// 2. Create and init BCITraining BCITraining _bciTraining = new BCITraining(); _bciTraining.Init();

// 3. connect to headset and then subscribe data. You need subscribe {"dev", "sys", "com", "fac"} List dataStreamList = new List(){DataStreamName.DevInfos, DataStreamName.MentalCommands, DataStreamName.SysEvents}; DataStreamManager.Instance.StartDataStream(dataStreamList, _headsetInformation.HeadsetID);

// 4. Query Profiles and load an existed profile. You might see the data at OnMentalCommandReceived()

// 5. Do some training ...` 2) For error"Invalid parameter" when you load profile. Please make sure the profile is not using other Apps (BCI ..) and please reference to doc[https://emotiv.gitbook.io/cortex-api/bci/setupprofile] If the issue still happen, please send us the request you sent to Cortex. Thanks

NarinderBrar commented 4 years ago

Thanks, Now its working

Following is the code

using System.Collections; using System.Collections.Generic; using UnityEngine; using EmotivUnityPlugin;

public class TestTraining : MonoBehaviour { BCITraining _bciTraining = new BCITraining();

void Start()
{
    _bciTraining.Init();
}

public void Sub()
{
    List<string> dataStreamList = new List<string>() { DataStreamName.DevInfos, DataStreamName.SysEvents, DataStreamName.MentalCommands };
    DataStreamManager.Instance.SubscribeMoreData(dataStreamList);
}

public void GetDetectionInfo()
{
    _bciTraining.GetDetectionInfo("mentalCommand");
}

public void QueryProfile()
{
    _bciTraining.QueryProfile();
}

public void LoadProfile()
{
    _bciTraining.LoadProfile("OpenMenu");
}

}