UI does not work in live setting but when replaying eeg signals

[Glolf]

Hello,

My colleague and I recently started working with timeflux. When I try to monitor my EEG signals from a MuseS headband, I don't see any graphs. But if I record the signals and replay them, it works as intended. My colleague and I run the same code, and it works for him. I run macOS Big Sur and have tested both firefox and chrome.

Am I doing something wrong, or what could be the reason?

Have a nice day!

//Frida

[mesca]

Hi Frida! Can you share the YAML file you are using?

[Glolf]

I add both the recording and replaying YAML below. In the UI I can see the numbers to the right (axis values) change which makes me think that the UI gets the data but simply not plot the lines in the real-time setting.

---

Recording graph

# Timeflux graph doing the following:
# - Reads an EEG lsl-stream of data
# - Makes some signal prcessing (Rolling + Welch + Bands)
# - Sends data to browser-UI
# - Reccords data to a .hdf5 file for later usage

graphs:

  # ===== The publish/subscribe broker graph =====
  #
  - id: PubSubBroker
    nodes:
    # Allow communication between graphs
    - id: Broker
      module: timeflux.nodes.zmq
      class: Broker

  # ===== The main processing graph =====
  #
  # Subscribes to the raw-eeg signals and makes some signal processing (Rolling
  # + Welch + Bands:alpha) to output (alpha-)band powers foe each electrode.
  #
  - id: Processing
    nodes:
    # Receive EEG signal from the network
    - id: LSL
      module: timeflux.nodes.lsl
      class: Receive
      params:
        prop: type
        value: EEG
    # Continuously buffer the signal
    - id: Rolling
      module: timeflux.nodes.window
      class: Window
      params:
        length: 1.5
        step: 0.5
    # Compute the power spectral density
    - id: Welch
      module: timeflux_dsp.nodes.spectral
      class: Welch
    # Average the power over band frequencies
    - id: Bands
      module: timeflux_dsp.nodes.spectral
      class: Bands
    # Publish the raw EEG signal
    - id: PublisherRaw
      module: timeflux.nodes.zmq
      class: Pub
      params:
        topic: raw
    # Publish the frequency bands
    - id: PublisherBands
      module: timeflux.nodes.zmq
      class: Pub
      params:
        topic: bands
    - id: display_before
      module: timeflux.nodes.debug
      class: Display
    # Connect nodes
    edges:
      - source: LSL
        target: Rolling
      - source: Rolling
        target: Welch
      - source: Welch
        target: Bands
      - source: LSL
        target: PublisherRaw
      - source: Bands:alpha
        target: PublisherBands
      # - source: LSL
      #   target: display_before
    # Run this graph 25 times per second
    rate: 4

  # ===== UI-Monitor Graph =====
  #
  # Subscribes to the raw-eeg signals and the real-time computed (alpha-)bands
  # from the processing node. Then these topics are sent to the UI-node (user
  # interface)
  #
  - id: MonitorGraph
    nodes:
    - id: sub
      module: timeflux.nodes.zmq
      class: Sub
      params:
        topics:
        - raw
        - bands
    - id: monitor
      module: timeflux_ui.nodes.ui
      class: UI
    - id: display_after
      module: timeflux.nodes.debug
      class: Display

    edges:
      - source: sub:raw
        target: monitor:raw
      - source: sub:bands
        target: monitor:bands
      - source: sub:raw # raw or bands
        target: display_after
    rate: 1

  # ===== The recorder graph =====
  #
  # Subscribes to the raw eeg-signals and the computed (alpha-)band powers and
  # saves the data as a .hfd5 file that can be replayed later on.
  #
  - id: SaveToFileGraph
    nodes:
    # Receive data streams from the broker
    - id: Subscriber
      module: timeflux.nodes.zmq
      class: Sub
      params:
        topics:
        - raw
        - bands
    # Record to file
    - id: Recorder
      module: timeflux.nodes.hdf5
      class: Save
      params:
        #path: /home/mgn/Documents/summer2021/Martin/timeflux/data
        path: /Users/frida/Documents/Forskning-GIT/summer2021/Martin/timeflux/data/
    # Connect nodes
    edges:
      - source: Subscriber:raw
        target: Recorder:eeg_raw
      - source: Subscriber:bands
        target: Recorder:eeg_bands
    # Update file every second
    rate: 1

Replay graph

# Timeflux graph doing the following:
# - Replays EEG-data corresponging (alpha-)band power from a .hdf5 file
# - Makes some signal prcessing on the EEG-data (Rolling + Welch + Bands)
# - Sends EEG-data, computed and precomputed (alpha-)band powers to browser-UI

graphs:

  # ===== The publish/subscribe broker graph =====
  #
  - id: PubSubBroker
    nodes:
    # Allow communication between graphs
      - id: broker
        module: timeflux.nodes.zmq
        class: Broker

  # ===== The Replay Graph/Node =====
  #
  # Replays the a previously reccorded .hdf5 file. In the particular file there
  # is both raw eeg-data (from MuseS) and corresponging (alpha-)bands that were
  # computed (Rolling + Welch + Bands:alpha) in real-time when reccording the
  # data
  #
  - id: ReplayGraph
    nodes:
    # Replay deta from hdf5 file.
      - id: replay
        module: timeflux.nodes.hdf5
        class: Replay
        params:
          #filename: /home/mgn/Documents/summer2021/Martin/timeflux/data/20210624-073132.hdf5
          filename: /Users/frida/Documents/Forskning-GIT/summer2021/Martin/timeflux/data/20210629-125703.hdf5
          keys:
            - /eeg/raw
            - /eeg/bands
  #          - /events
    # Publisher eeg
      - id: pub_eeg
        module: timeflux.nodes.zmq
        class: Pub
        params:
          topic: eeg
    # Publisher precomputed bands (Obs: only aplha-band)
      - id: pub_bands
        module: timeflux.nodes.zmq
        class: Pub
        params:
          topic: bands_precompute
    # Display node
      - id: display
        module: timeflux.nodes.debug
        class: Display

    edges:
    - source: replay:eeg_raw # This have to macth whatever name we gave when revording the data: Recorder:eeg_raw
      target: pub_eeg
    - source: replay:eeg_bands # This have to macth whatever name we gave when revording the data: Recorder:eeg_bands
      target: pub_bands
#    - source: replay:eeg_bands # This have to macth whatever name we gave when revording the data: Recorder:eeg_bands
#      target: display
    rate: 0

  # ===== The main processing graph =====
  #
  # Subscribes to the replayed raw-eeg signals and makes the signal processing
  # (Rolling + Welch + Bands:alpha) to get the same output as the prereccorded
  # (alpha-)bands.
  #
  - id: Processing
    nodes:
    # Subscriber node
    - id: subscriber
      module: timeflux.nodes.zmq
      class: Sub
      params:
        topics:
        - eeg
    # Continuously buffer the signal
    - id: Rolling
      module: timeflux.nodes.window
      class: Window
      params:
        length: 1.5
        step: 0.5
    # Compute the power spectral density
    - id: Welch
      module: timeflux_dsp.nodes.spectral
      class: Welch
    # Average the power over band frequencies
    - id: Bands
      module: timeflux_dsp.nodes.spectral
      class: Bands
    # Publish the frequency bands
    - id: PublisherBands
      module: timeflux.nodes.zmq
      class: Pub
      params:
        topic: bands
    - id: display_before
      module: timeflux.nodes.debug
      class: Display
    # Connect nodes
    edges:
      - source: subscriber:eeg
        target: Rolling
      - source: Rolling
        target: Welch
      - source: Welch
        target: Bands
      - source: Bands:alpha
        target: PublisherBands
      # - source: subscriber:eeg
      #   target: display_before
#      - source: Welch # Rolling #Bands:alpha
#        target: display_before
    # Run this graph X times per second
    rate: 4

  # ===== UI-Monitor Graph =====
  #
  # Subscribes to the replayed raw-eeg signals and replayed precomputed (alpha-)
  # bands, as well as the real-time computed (alpha-)bands from the processing
  # node. Then these topics are sent to the UI-node (user interface)
  #
  - id: MonitorGraph
    nodes:
    - id: sub
      module: timeflux.nodes.zmq
      class: Sub
      params:
        topics:
        - eeg
        - bands
        - bands_precompute
    - id: monitor
      module: timeflux_ui.nodes.ui
      class: UI
    - id: display_after
      module: timeflux.nodes.debug
      class: Display

    edges:
      - source: sub:eeg
        target: monitor:eeg
      - source: sub:bands_precompute # raw or bands
        target: monitor:bands_precompute
      - source: sub:bands # raw or bands
        target: monitor:bands
#      - source: sub:bands
#        target: monitor:bands
      - source: sub:bands_precompute # raw or bands
        target: display_after
    rate: 1

[mesca]

It is probably a problem with timestamping. You can quickly check if this is the case by plugging a Display node after your EEG node. The indices should be UTC timestamps.

If this is not the case, you probably need to pass additional parameters to synchronize timestamps. If you are acquiring the data from another computer, you need to set the sync parameter to network. I guess you are using MuseLSL, but make sure that the timestamps sent from the source are seconds. Otherwise, you will need to set the unit parameter. Check the documentation for details about the parameters.

Also, check that you have the latest Timeflux version, as bug fixes were applied on the LSL module in March.

In replay mode, timestamps are resynchronized to local time in order to simulate a real-time application. This is why you can see the signal when replaying but not when recording.

As an alternative, you may want to try the Brainflow plugin, which seems to now have a native Muse support (not tested). I just released a new version with the updated Brainflow lib.

Let me know how it goes.

[Glolf]

Thank you! I will get back to you when I have tested!

[Glolf]

Okay, I have now identified the problem, but I don't know what to do about it. I have the latest version of timeflux and pylsl.

As you say, I use MuseLSL, which sends timestamps in UTC seconds, e.g., 1625491694.484428.

In the LSL node in timeflux, there is one part where the timestamps are translated to pandas timestamps stamps = pd.to_datetime(stamps, format=None, unit=self._unit), from this I still get correct timestamps, e.g., 2021-07-05 13:28:14.484427929.

What then happens is that the timestamps are adjusted by the offset stamps += self._offset, and this is where it goes wrong because I get: '2073-01-06 19:16:37.586437785'. The problem is due to the call to pylsl.local_clock() - because from this, I get 28293.923076561, but it should be something closer to time.time(). My offset is basically time.time(). I compared it to my college: he got something more reasonable from pylsl.local_clock(), and the plotting works for him.

[mesca]

I haven't used MuseLSL for a while. I thought it was using the LSL local clock to set the timestamps, but it seems to set them manually. Please try to add this param to your LSL node: sync: null.

[Glolf]

Woohoo! It works!! Thank you, Pierre!

As I track it, it seems they are using time.time() to set the times if not otherwise specified.