search

OpenBCI / OpenBCI_GUI

A cross platform application for the OpenBCI Cyton and Ganglion. Tested on Mac, Windows and Ubuntu/Mint Linux.
MIT License
707 stars 256 forks source link

OpenBCI GUI App runs but OpenBCI GUI on Processing fails (OSX) #264

Closed gerrievanzyl closed 6 years ago

gerrievanzyl commented 6 years ago

When I download and run the new (3.0) OpenBCI_GUI app for OSX it opens and runs.

When I download the latest OpenBCI_GUI sketch from here (Master) https://github.com/OpenBCI/OpenBCI_GUI

And run the sketch from Processing it does not progress past the point where I click on the Cyton board.

Have I missed something?

Here is the output from the terminal:

Launching application from local data dir OpenBCI_GUI: hubStart: OS Detected: Mac Welcome to the Processing-based OpenBCI GUI! Last update: 9/5/2016 For more information about how to work with this code base, please visit: http://docs.openbci.com/OpenBCI%20Software/ Graphics & GUI Library: ControlP5 2.2.6 infos, comments, questions at http://www.sojamo.de/libraries/controlP5 attempting to load images for presentation... DONE loading images! java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at java.net.Socket.connect(Socket.java:538) at java.net.Socket.(Socket.java:434) at java.net.Socket.(Socket.java:211) at processing.net.Client.(Unknown Source) at OpenBCI_GUI$Hub.startTCPClient(OpenBCI_GUI.java:12101) at OpenBCI_GUI$Hub.(OpenBCI_GUI.java:12077) at OpenBCI_GUI.systemUpdate(OpenBCI_GUI.java:927) at OpenBCI_GUI.draw(OpenBCI_GUI.java:428) at processing.core.PApplet.handleDraw(PApplet.java:2437) at processing.opengl.PSurfaceJOGL$DrawListener.display(PSurfaceJOGL.java:849) at jogamp.opengl.GLDrawableHelper.displayImpl(GLDrawableHelper.java:692) at jogamp.opengl.GLDrawableHelper.display(GLDrawableHelper.java:674) at jogamp.opengl.GLAutoDrawableBase$2.run(GLAutoDrawableBase.java:443) at jogamp.opengl.GLDrawableHelper.invokeGLImpl(GLDrawableHelper.java:1293) at jogamp.opengl.GLDrawableHelper.invokeGL(GLDrawableHelper.java:1147) at com.jogamp.newt.opengl.GLWindow.display(GLWindow.java:759) at com.jogamp.opengl.util.AWTAnimatorImpl.display(AWTAnimatorImpl.java:81) at com.jogamp.opengl.util.AnimatorBase.display(AnimatorBase.java:452) at com.jogamp.opengl.util.FPSAnimator$MainTask.run(FPSAnimator.java:178) at java.util.TimerThread.mainLoop(Timer.java:555) at java.util.TimerThread.run(Timer.java:505) Instantiating hub object...

andrewjaykeller commented 6 years ago

You need to follow the instructions found here to add the hub to your data folder.

gerrievanzyl commented 6 years ago

AJ,

Thanks so much. Go the wifi shield working!!! Looks amazing. I want to work with my daughters to contribute to the code to enable the insertion of "stimulation" markers into the data stream. However, we are going to need your guidance here. Before I get to some questions.

Here is a little "feature" you may want to know about.

  1. When the OpenBCI.app was in my "Downloads" folder it kept hanging when I clicked on the "Start System" button. Only when I moved it to Applications did it work correctly.

OK, onto where we need your help.

Getting git set up correctly:

  1. We forked your new https://github.com/OpenBCI/OpenBCI_GUI repository in github
  2. Then we downloaded it to our local machine
  3. We can now go ahead and edit the code on our local machine

The process to make changes (just eyeball to check that we are doing this right):

  1. On our local machine we select the right branch (I imagine it would be "Dev" - correct?)
  2. Then we edit the code
  3. Every so often we "Commit" our changes
  4. When we are ready (or think we are) we push our changes from our local machine to our github account
  5. Then from our github account we issue a "pull request to you"

If this is all correct let's get into the details of how our system works in OpenBCI V2

  1. We set up a UDP listener on the OpenBCI software using the hypermedia.net library that you already use. The listener is on port
  3. To this listener we attach a function that processes incoming UDP packets.
  4. When the system is started we put the board into the "marker mode". This is the same mode we have already implemented in the latest board firmware. On OpenBCI V2 this was done by sending "\4"
    1. You may recall that on the firmware this disables the accelerometers and uses the Accel X channel for the markers

When the system is running the following happens

  1. When OpenBCI receives a UDP message on port 51000 it passes the packet onto the UDP handler function. This function receives a packet string (something like "STIM9") it validates the packet and pulls out the number on the end (in this case the 9)
  2. The handler writes "`9" to the serial stream (using the write() function)
  3. The board firmware receives the characters on the serial stream and parses the message and inserts the value 9 into the data stream in the Accel X field.

We have tried to make these changes to 3.0 but it seems that you have made some changes to the OpenBCIHub does not allow our messages to go through.

  1. It gives an error when we try to put the board into "Marker mode" ("\4")
  2. Similarly it gives an error when we send the marker signal ("`n" - where N is number 0-9)

We don't really understand JS but there seems to be a "parse" function that must be configured to pass the messages on to the OpenBCI board.

This is what we propose:

  1. We believe the changes we are proposing (together with your new wifi shield) will be really, really exciting for OpenBCI as it will allow the headset to be used for many more research applications.
  2. Can you make the changes to OpenBCIHub to let these messages through (or tell us what we are doing wrong)?
  3. We will make the other changes and after we have tested the system we will send you a pull request with the changes.
  4. To make sure we are making the changes correctly we want to confirm the correct places to implement our changes:
    1. We do the UDP initialization in the very end of the setup() function in the main OpenBCI_GUI.pde
    2. We added the UDP event handler to the OpenBCI_GUI.pde (after setup and before the draw() loop) - tell us where we should put this
    3. We added the board mode to the public void startDataTransfer() function in the BoardCyton.pde file
  5. Finally, we probably need to add one extra selection to the GUI before "Start System" to allow the user to set the Board mode (Accel, TimeSync, Marker, etc) Unfortunately, we have very little experience in GUI stuff so I would be very nervous doing that.

Let me know whether you could help us on this.

Regards Gerrie

On Mon, Oct 16, 2017 at 10:58 AM, AJ Keller notifications@github.com wrote:

You need to follow the instructions found here http://docs.openbci.com/OpenBCI%20Software/01-OpenBCI_GUI#the-openbci-gui-running-the-openbci-gui-from-the-processing-ide-install-openbci-hub-on-mac to add the hub to your data folder.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/OpenBCI/OpenBCI_GUI/issues/264#issuecomment-336913682, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSh_4ICtkYWTDtyfDPkADsWnwYBvaJ0ks5ss28JgaJpZM4P505o .

andrewjaykeller commented 6 years ago
  1. When the OpenBCI.app was in my "Downloads" folder it kept hanging when I clicked on the "Start System" button. Only when I moved it to Applications did it work correctly.

What is the exact operating system version you are using? I'm chasing this bug in #270

  1. On our local machine we select the right branch (I imagine it would be "Dev" - correct?) Yes, always use development branch git checkout development
  2. Then we edit the code
  3. Every so often we "Commit" our changes i commit anytime i code something I don't want to loose, so commit frequency changes, but generally I don't leave valuable code uncommitted overnight
  4. When we are ready (or think we are) we push our changes from our local machine to our github account. push at least once a day
  5. Then from our github account we issue a "pull request to you". yup! submit a pull request to the OpenBCI_GUI/development branch

Always run master when getting started, but develop from the development branch

andrewjaykeller commented 6 years ago

Moving into the implementation:

  1. It gives an error when we try to put the board into "Marker mode" ("\4")

Who is it? What is the exact firmware version you are using? Please try again with 3.1.0 32bit firmware which solves some stability issues.

Can you make the changes to OpenBCIHub to let these messages through (or tell us what we are doing wrong)?

Please use the cyton.write(String) function to send commands to the connected board. You can pass a string with the /4 which will properly be routed to the board.

We will make the other changes and after we have tested the system we will send you a pull request with the changes.

Sure. This sounds great, I will make comments on spacing (please try to adhere to the spacing you see in the file BoardCyton.ino for example)

We added the UDP event handler to the OpenBCI_GUI.pde (after setup and before the draw() loop) - tell us where we should put this

So at a big picture, you want to send STIM9 to the UDP port 51000, which then sends the 9, in board marker format `9. Is that correct? Maybe we should put the UDP listener in the OpenBCIHub, NodeJS is made for asynchronous server actions such as this use case. Processing, not so much.

andrewjaykeller commented 6 years ago

Finally, we probably need to add one extra selection to the GUI before "Start System" to allow the user to set the Board mode (Accel, TimeSync, Marker, etc) Unfortunately, we have very little experience in GUI stuff so I would be very nervous doing that.

I love this idea and opened #271

gerrievanzyl commented 6 years ago

OSX 10.13 (High Sierra)

On Mon, Oct 23, 2017 at 1:09 PM, AJ Keller notifications@github.com wrote:

  1. When the OpenBCI.app was in my "Downloads" folder it kept hanging when I clicked on the "Start System" button. Only when I moved it to Applications did it work correctly.

What is the exact operating system version you are using?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/OpenBCI/OpenBCI_GUI/issues/264#issuecomment-338625279, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSh_8EejTCSCs7pKHJQQv-yedHOnznZks5svHPNgaJpZM4P505o .

gerrievanzyl commented 6 years ago

AJ,

You are right, architecturally, UDP listener should go into the OpenBCI HUB.

This was exactly the kind of feedback that is really useful before we jump in.

I will have to check out the JS code that makes up hub - ball back in my court.

BTW: You will experience an issue with hub as you look at the GUI that sets the board mode.

When the GUI sends a cyton.write(String) the hub, the hub seems to intercept it and parse it before forwarding it on.

When the hub receives "/4" it throws and error because the parser does not know what "/" means.

Similarly, in previous versions of the hub the "`x" string would be passed through to the board without any changes, but now it causes an parser error on the hub.

Does this make sense?

If not, let me have a look again and collect more information.

Regards Gerrie

On Mon, Oct 23, 2017 at 1:29 PM, AJ Keller notifications@github.com wrote:

Moving into the implementation:

  1. It gives an error when we try to put the board into "Marker mode" ("\4")

Who is it? What is the exact firmware version you are using? Please try again with 3.1.0 32bit firmware https://github.com/OpenBCI/OpenBCI_32bit_Library/releases/tag/v3.1.0 which solves some stability issues.

Can you make the changes to OpenBCIHub to let these messages through (or tell us what we are doing wrong)?

Please use the cyton.write(String) function https://github.com/OpenBCI/OpenBCI_GUI/blob/development/OpenBCI_GUI/BoardCyton.pde#L346-L352 to send commands to the connected board. You can pass a string with the /4 which will properly be routed to the board.

We will make the other changes and after we have tested the system we will send you a pull request with the changes.

Sure. This sounds great, I will make comments on spacing (please try to adhere to the spacing you see in the file BoardCyton.ino for example)

We added the UDP event handler to the OpenBCI_GUI.pde (after setup and before the draw() loop) - tell us where we should put this

So at a big picture, you want to send STIM9 to the UDP port 51000, which then sends the 9, in board marker format `9. Is that correct? Maybe we should put the UDP listener in the OpenBCIHub, NodeJS is made for asynchronous server actions such as this use case. Processing, not so much.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/OpenBCI/OpenBCI_GUI/issues/264#issuecomment-338629488, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSh_0dObFQxDSgCABwJhoK3WJl-kC6oks5svHh-gaJpZM4P505o .

gerrievanzyl commented 6 years ago

AJ,

Quoted from a previous conversation:

Can you make the changes to OpenBCIHub to let these messages through (or tell us what we are doing wrong)?

AJ: Please use the cyton.write(String) function https://github.com/OpenBCI/OpenBCI_GUI/blob/development/OpenBCI_GUI/BoardCyton.pde#L346-L352 to send commands to the connected board. You can pass a string with the /4 which will properly be routed to the board.

I struggled with this and in end I jumped into the code and found that cyton.write(String) https://github.com/OpenBCI/OpenBCI_GUI/blob/development/OpenBCI_GUI/BoardCyton.pde#L346-L352 caused the "/4" string to be rejected by the Hub (unrecognizedCommand(client, kTcpCmdError, msgElements[0]) in background.js). It seems that the correct call is Hub.sendCommand(String) .

This method does the formatting of the String into the Hub command format and passes it to the Wifi.prototype.write({String, Array, etc}) method.

I am back in the US and looked at this over the weekend. I want to go step for step here - baby steps!

  1. Does your OpenBCI firmware correctly send the accelerometer information? It does not look like my setup is sending this info throug

I see you are making some changes. Is there any chance that we can talk briefly over the phone? A few minutes of your help will really help me.

Text me your number on 67888344444

Gerrie

On Mon, Oct 23, 2017 at 7:29 AM, AJ Keller notifications@github.com wrote:

Moving into the implementation:

  1. It gives an error when we try to put the board into "Marker mode" ("\4")

Who is it? What is the exact firmware version you are using? Please try again with 3.1.0 32bit firmware https://github.com/OpenBCI/OpenBCI_32bit_Library/releases/tag/v3.1.0 which solves some stability issues.

Can you make the changes to OpenBCIHub to let these messages through (or tell us what we are doing wrong)?

Please use the cyton.write(String) function https://github.com/OpenBCI/OpenBCI_GUI/blob/development/OpenBCI_GUI/BoardCyton.pde#L346-L352 to send commands to the connected board. You can pass a string with the /4 which will properly be routed to the board.

We will make the other changes and after we have tested the system we will send you a pull request with the changes.

Sure. This sounds great, I will make comments on spacing (please try to adhere to the spacing you see in the file BoardCyton.ino for example)

We added the UDP event handler to the OpenBCI_GUI.pde (after setup and before the draw() loop) - tell us where we should put this

So at a big picture, you want to send STIM9 to the UDP port 51000, which then sends the 9, in board marker format `9. Is that correct? Maybe we should put the UDP listener in the OpenBCIHub, NodeJS is made for asynchronous server actions such as this use case. Processing, not so much.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/OpenBCI/OpenBCI_GUI/issues/264#issuecomment-338629488, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSh_0dObFQxDSgCABwJhoK3WJl-kC6oks5svHh-gaJpZM4P505o .

gerrievanzyl commented 6 years ago

AJ,

I have made some changes to OpenBCI_32Bit firmware. I noticed that the development fork is really old - it does not contain much of the stuff I worked on before. I have forked off master. Since I really don' t understand git - I trust that you will be able to merge the changes somehow.

More questions:

  1. In your code I see do quite extensive print() statements to the wifi stream. How can I view what comes back? I suspect I need to do something with the Hub to see these messages
  2. Can I use the Python API with the wifi shield?
  3. It looks like the SD_Card_Stuff does not make allowance for file sizes when writing at higher sampling rates. For example 15 minutes at 250Hz is only ~4 minutes at 1000Hz. The change would be really easy - would you like me to make that?
  4. Finally, I have a favor to ask you. I am working with two high school students who are using the OpenBCI for a very impressive science fair project. They spent some time with Teon Brooks an got some great feedback from him - I see you know him well. It was just after Teon relocated to the west coast and he was really under the hammer. Could you spare a few minutes to listen to what they are doing and give some input? We could do the session over skype or hangouts

Thanks for your help Gerrie

On Tue, Oct 31, 2017 at 10:47 AM, Gerrie van Zyl gerrie@van-zyl.net wrote:

AJ,

Quoted from a previous conversation:

Can you make the changes to OpenBCIHub to let these messages through (or tell us what we are doing wrong)?

AJ: Please use the cyton.write(String) function https://github.com/OpenBCI/OpenBCI_GUI/blob/development/OpenBCI_GUI/BoardCyton.pde#L346-L352 to send commands to the connected board. You can pass a string with the /4 which will properly be routed to the board.

I struggled with this and in end I jumped into the code and found that cyton.write(String) https://github.com/OpenBCI/OpenBCI_GUI/blob/development/OpenBCI_GUI/BoardCyton.pde#L346-L352 caused the "/4" string to be rejected by the Hub (unrecognizedCommand(client, kTcpCmdError, msgElements[0]) in background.js). It seems that the correct call is Hub.sendCommand(String) .

This method does the formatting of the String into the Hub command format and passes it to the Wifi.prototype.write({String, Array, etc}) method.

I am back in the US and looked at this over the weekend. I want to go step for step here - baby steps!

  1. Does your OpenBCI firmware correctly send the accelerometer information? It does not look like my setup is sending this info throug

I see you are making some changes. Is there any chance that we can talk briefly over the phone? A few minutes of your help will really help me.

Text me your number on 67888344444

Gerrie

On Mon, Oct 23, 2017 at 7:29 AM, AJ Keller notifications@github.com wrote:

Moving into the implementation:

  1. It gives an error when we try to put the board into "Marker mode" ("\4")

Who is it? What is the exact firmware version you are using? Please try again with 3.1.0 32bit firmware https://github.com/OpenBCI/OpenBCI_32bit_Library/releases/tag/v3.1.0 which solves some stability issues.

Can you make the changes to OpenBCIHub to let these messages through (or tell us what we are doing wrong)?

Please use the cyton.write(String) function https://github.com/OpenBCI/OpenBCI_GUI/blob/development/OpenBCI_GUI/BoardCyton.pde#L346-L352 to send commands to the connected board. You can pass a string with the /4 which will properly be routed to the board.

We will make the other changes and after we have tested the system we will send you a pull request with the changes.

Sure. This sounds great, I will make comments on spacing (please try to adhere to the spacing you see in the file BoardCyton.ino for example)

We added the UDP event handler to the OpenBCI_GUI.pde (after setup and before the draw() loop) - tell us where we should put this

So at a big picture, you want to send STIM9 to the UDP port 51000, which then sends the 9, in board marker format `9. Is that correct? Maybe we should put the UDP listener in the OpenBCIHub, NodeJS is made for asynchronous server actions such as this use case. Processing, not so much.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/OpenBCI/OpenBCI_GUI/issues/264#issuecomment-338629488, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSh_0dObFQxDSgCABwJhoK3WJl-kC6oks5svHh-gaJpZM4P505o .

micuat commented 6 years ago

I think it is more user friendly if this exception is caught and output the link to console or window