Processing - C Live Demo

[dymnz]

RMSDown: Processing \ REC: Processing \ FE: Processing \ NN: C \ DIS: Processing

Processing TODO

• Feature Extraction
  • [x] Continous windowed RMS
  • [x] Demix (Matrix mult.)
  • [x] LPF
  • [x] Downsample
  • [x] Normalization
  • [x] Serial output MO
  • [x] Serial input SI

C TODO

• NN Inference
  • [x] Write/Load trained model
  • [x] NN inference step
  • [x] Serial input MI
  • Slow serial com speed (~5Hz --> 35Hz) is fixed by changing serial read in C from 1-byte to 24-byte
    • No initial sync up, may cause problem if serial buffer is not cleared
  • [x] Serial output SO

[dymnz]

Serial Comm

Setup

• socat -d -d pty,raw,echo=0 pty,raw,echo=0
  • Connect to /dev/pts/N1, /dev/pts/N2 for virtual serial port
  • To send via shell: echo "test" > /dev/pts/N1
  • To receive via shell: cat < /dev/pts/N2

Processing

• Data in from Arduino
• Data out to NN_1
• Data out to NN_2
• Data in from NN_1
• Data in from NN_2

NN

• Data in from Processing
• Data out to Processing

[dymnz]

Detail on Processing

Procedure

• Get sEMG from Ard
• Receive angle from NN
• RMS
• TDSEP
• LPF
• Downsample
• Send sEMG to NN
• Receive angle from NN
  • Suppose NN is fast enough so that no sEMG is sent when NN is still processing
• Plot angle from NN
• Plot angle from GT

Note on GT/NN angle

• GT angle received sparsely from Ard
• 1 sEMG data sent to NN --> 1 angle data received from NN

Syncing GT angle and NN angle

• GT angle and NN angle should be synced w/ sEMG
• GT angle isn't drawn before NN angles are received for the same buffer_index
• RMSE is calculated within a window

[dymnz]

Note on Demo

• Recording of training set
  • Varying level
  • Mixed FESP
  • [x] Try continuous movement
  • w/ 35SPS, 10 sec per segment is acceptable
    • 10SPS is too low for live demo
  • 35SPS, 10sec segment w/ 1min training example yields OK result
    • OK FLX/EXT
    • Bad PRO/SUP as SUP also activates for FLX
• Copy demix matrix / normalization value from matlab to Processing
• Train NN and find the best model
  • 6 HN for Roll / 4 HN for Pitch

Observation

• FLX/EXT very stable and accurate
• PRO/SUP unstable, SUP value only show one
  • May need to fix data partition method

TODO

• [x] Generate FLX/EXT/PRO/SUP combined dataset
  • Processed sample for NN training should be spliced when |Roll| or |Pitch| > 10
  • 2 separate w/ 2 angle (Roll/Pitch) for 2 NN
• [x] Port dry_run code back to live_run code

[dymnz]

Dry run

• NNs are trained w/ previously processed samples
• NNs are loaded w/ previously trained model

Procedure

• Processing read samples from past recording
• Processing send the processed sample to NNs
• NNs send the inference result back
• Processing plot the result and calculate the RMSE

Note

• Only difference between this and actual demo is the source of the sEMG
• Processed sample for NN training should be spliced when |Roll| or |Pitch| > 10

TODO

• [x] Processing sEMG recording read