search

BasedHardware / omi

AI wearables
https://omi.me
MIT License
3.61k stars 444 forks source link

Integrate glasses to Friend's app ($2000) #222

Closed kodjima33 closed 3 months ago

kodjima33 commented 4 months ago

Task: Integrate openglass with Friend's app with wearable

Iti should send images and create conversations same way our audio wearable does it.

"memories" should also change to include at least one image from the event

Image

This is a paid task. Reward is $2000 in cash. Simply link your PR with this task and we will check it asap

Sagar1497 commented 4 months ago

Hello Team ,

I'd like to propose some improvements to enhance project's functionality. Currently, we have two separate wearable devices, but I believe integrating them into the mobile app can:

  1. we have the hardware setup already
  2. Software setup : Developing the Friend's firmware using an IDE (Arduino IDE or PlatformIO) and libraries: speech recognition (Espressif ASR or cloud-based API) and Bluetooth Low Energy (BLE) for communication. The code can captures audio, converts it to text, formats data, and transmits it to the app via Bluetooth. and I guess this has been developed already

The OpenGlasses leverage the same development environment as the FRIEND. Specific camera libraries and Bluetooth (BLE) functionalities are needed. The code captures images (on intervals or user command), optionally performs basic processing (like adjusting brightness), compresses the image data, and transmits it with timestamps to the smartphone app via Bluetooth.

3.Mobile Application Development : Functionality of the app involves : Establish bluetooth connection , receiving and interpreting text data , here the new thing is the pictures which is been captured is displayed in smartphone application instead of computer [As shown in promotional video] which by far becomes best and easy way to create a memories section where the data will be saved and gives us details like calories data and name of person and so on... To avoid overloading the smartphone's storage, the app implements functionalities like automatic deletion of older images or user control over deleting images associated with specific memories.

  1. Mobile application as central hub.
Joven-G commented 3 months ago

I can do it as I mentioned to you in discord. let's connect with me.

francip commented 3 months ago

I started some work for this. On the OpenGlass firmware side, I have https://github.com/BasedHardware/OpenGlass/pull/40 to enable Device Information Service, so that the app can easily distinguish between Friend and OpenGlass, and different firmware versions. On the Friend app side, I'll have in couple of days base change that enables the app to discover and connect to OpenGlass, albeit the audio streaming doesn't work yet properly. At least this will enable folks that want to work on enabling the pictures to start working.

francip commented 3 months ago

@josancamon19 Btw, I started on Opus implementation on OpenGlass, so we should try to speed Opus in the app.