Connect app with Frame by Brilliant Labs ($2000)

[josancamon19]

Is your feature request related to a problem? Please describe. https://docs.brilliant.xyz/frame/building-apps/

In the connect devices page, the BLE characteristic of brilliant labs should appear. We should be able to connect to it from the same screen. once connected, it should start streaming photos every 10 seconds as open glass does, maybe even faster as they support 3fps.

As they support audio and photo streaming, we should be able to retrieve both of them, and create memories from it.

• Battery level should be correct.
• Backend source should be brilliantlabs-frame
• Backend should process audio and photo for creating the memory and indexing it.

It should operate then as a friend/openglass in a single device, having audio and images displayed + processed in the app.

Transcript logic should be the same, just streaming the received bytes in the transcript websocket.

[kodjima33]

Increased bounty to $2k

[OkGoDoIt]

I discussed this with @kodjima33 today. I can take a first pass at this. A few questions that would help me understand this better:

Is there currently any code in the Friend app that handles images or is integrating that into the pipeline part of the scope? If there is no image pipeline yet, then a first pass Frame integration should be audio-only so as to keep complexity down.

Is there currently any way to select your preferred device as a user of the app? Is it expected that it will just connect to the nearest Friend or Frame device or is it expected to guide the user through setting up their device? If there is not currently a UX for choosing a device, then this first pass will just try to connect to the closest device, but that may not be ideal.

Looking through the current app code, it seems to assume it's talking to a Friend device. If the plan is to support multiple devices in the future, it's probably better to encapsulate specific device functionality in some sort of generic device base class. I'll need a better understanding of planned 3rd party device support to understand how high-level this base class should be.

My Flutter experience is minimal, so I may need to work with you all on this once I've got some basics started. Nik has my contact info, and I'll plan to come to your office Thursday.

[francip]

If there was no device previously connected, the app will list all devices it can connect to and the user will choose which one. Currently that list supports Friend and OpenGlass devices.

[kodjima33]

@OkGoDoIt

Is there currently any code in the Friend app that handles images or is integrating that into the pipeline part of the scope?

Yes there is already existing image functionality for Openglass. We've tested it and it was working. As far as I know, it's currently commented out as we were moving to backend

Is there currently any code in the Friend app that handles images or is integrating that into the pipeline part of the scope? If there is no image pipeline yet, then a first pass Frame integration should be audio-only so as to keep complexity down.

yes, Franci responded before

it's probably better to encapsulate specific device functionality in some sort of generic device base class

if you could do it this way, you would be a legend! we plan to support 10+ devices.

[OkGoDoIt]

Awesome, thank you. Sorry if those questions were a bit basic but I’m still exploring the codebase.

[josancamon19]

@Becca-Saka