Mobility - GLBOX shrink or dissapear

[CaydenPierce]

An issue with the usability, we arability, and mobility of the system is the need to carry the GLBOX laptop around everywhere.

It's heavy, bulky, requires a bag to carry, and thus isn't highly mobile.

There are two main options here (please feel free to suggest other options):

1. Move the GLBOX to a smaller wearable that fits in pocket

   • all SBCs that I know of which have compute are too big (i.e. RaspberryPi 4 is no better than laptop as it won't fit in pocket)
   • Running GNU/Linux on Android phone hardware possible but buggy, unsupported, probably cause a lot of tangential issues
   • Pi Zero W (or similiar GNU/Linux SBCs) are small, low power, and could do it, but they are so computationally weak that we would have to stream video to the cloud or the ASP anyway for any intense computer vision

2. Move GLBOX to the cloud

   • requires implementing audio streaming from ASG to GLBOX over internet
   • introduces latency on transcription and command response
   • will use a lot of data, means the system will now only work with an internet connection
   • better long term - means that users don't have to buy 2 pieces of hardware, and unlimited, high speed data is a reasonable assumption for 5 years out

Decision

2, move GLBOX to the cloud

Why?

For a number of reason, #2 makes more sense:

• requiring another piece of hardware is a huge deterrent to use
  • expensive
  • another thing to carry
  • another thing to keep charged
  • increased system complexity and mental load
• no current mobile GNU/Linux implementation is all three of:
  1. Powerful enough to run heavy compute at small size and power
  2. Proper form factor to fit in a pocket (smartphone form factor)
  3. Reliable -in the future this may be possible with mobile GNU/Linux system (running GNU/Linux on mobile chipsets - current implementations (Ubuntu Touch, PostmarketOS, etc.) are not ready for production
• cloud implementation is more agile, can constantly make changes/push features/etc.

How?

The GLBOX code is already a Python socket server running in GNU/Linux, so moving it to the "the cloud" is almost already done. We just need to modify a few things to make it work on a cloud server as opposed to a wearable server.

MVP

• [x] audio capture on ASG that is streamed to the GLBOX (as opposed to current scheme where audio streams direct to GLBOX via Bluetooth)
• [x] finish implementing "heart beats" (pings) on both GLBOX and ASG sides (almost done)
• [x] switch from current LAN advertising scheme to referencing GLBOX with domain name

Further

• [ ] finish implementing a streaming VAD which will turn off speech to text service when there is no voice, turn speech to text back on when there is a voice, and will hold a circular buffer of audio so no transcriptions are lost

[CaydenPierce]

Almost complete at commit id c92e8e3ad23c2394bd661f0967628ee9255e647d working with @stairs1

TODO

• [x] fix transcription cloud socket issue - ASG Audio socket
• [x] encrypt audio
• [ ] implement VAD to start and stop STT stream
• [x] fix ASG AudioService failing if activity is stopped and restarted

[CaydenPierce]

Everything required for full move to the cloud is complete... except for VAD.

For VAD we are using WebRTCVad ... it works terribly, seemingly any environmental noise will trick it. This is not worth the time, will just make the experience worse. Thus, VAD is 90% complete being integrated but we are going to leave it for now as a future feature.

Commit d4cbebb18d779d759b1f43b564d7f1c56aefc498 marks the completion of this issue, includes fixes for bugs (server restart), and all new cloud features.

That means this Issue is complete. Thanks @stairs1 for help on GLBOX side