aegray / openfocals

Replacement Android app for Focals smart glasses (originally by North)
17 stars 9 forks source link

displaying single image png's #3

Open LaughingMaann opened 3 years ago

LaughingMaann commented 3 years ago

according to "Feature todo list" where [x] Custom app handler (requires installing software to glasses) [x] File transfer to focals (for icons, media) does this imply that the capability to push test images (file transfer media) is pre-built into the latest build at http://www.aegray.com/openfocals/openfocals.apk ?

(currently struggling to load test images for viewing through the north focal display using your app). otherwise, would be have to install an external ESFFileExplorer-cn.apk or Gallery.apk in the app manager setup in the glasses ?

may i very kindly ask for instructions on how to proceed? the apk is installed on my phone, but no option in the app's tab openfocals->Developer Tools

LaughingMaann commented 3 years ago

i did more browsing and i can see that there's a "phone screen mirroring to glasses screen" with the screen recorder service. how would one enable this too via the app? https://github.com/aegray/openfocals/commit/48d038123af3b9d54c253496a0686471d4f01a06

aegray commented 3 years ago

There is functionality built into the underlying source of the app to send files, however I don't think anything is exposed for actual use (I was using this to test out sending update packages to the focals). It would be an easy addition, however it's sort of painful to actually view a png without a custom app, unless it's just as an icon on a message. There's a hidden MediaPlayer experience that you could view a single png with but it's semi painful to get to. Let me know what you're trying to do here

To enable phone screen mirroring (or any custom app) is slightly more painful - it requires building a connector cable, doing some magic to allow you to add your own software, then installing some custom software on the glasses. If you're interested in going down that path, let me know and I can provide more info, but it's not a simple process.

LaughingMaann commented 3 years ago

the goal here with using your toolkit are the following 2 objectives, with option A plan (but would like more feedback): goal 1) load an image (png, jpg, bmp if need be) into the AR glass display, such that the image encompasses the entire field of viewing area and doesnt change goal 2) configure timeout so glasses dont go into sleepmode and change the display to some default screensaver (ie, make step 1's image display go away) option A) if the functionality to send files to the app is there, would it be less work then to expose that for developer use (and then view png with a custom app )? less work relative to option b below option B) enable phone screen mirroring with the existing ScreenRecordingService class in /focals_buddy/app/src/main/java/com/openfocals/services/network/present/providers/StreamingTextPresentationProvider.java

aegray commented 3 years ago

I think to do what you want (display an image on the glasses screen) - you're going to need to go the harder route and build a connector cable / install apps - I couldn't find any other easy way. Just sending a file won't do anything for you.

If you decide to go that route, I think the screen mirror app does 90% of what you want already - it disables sleep mode when its enabled and continuously displays whatever the last image sent from the glasses app was (can get up to ~10-15 frames / second).

The hard part is getting custom software installed - if you want to discuss that more, log on discord at https://discord.gg/VKgeFjzQ

The easy part would be modifying the phone software:

The current code that is grabbing screen images, formatting them, and sending them to the ScreenMirrorWSService is: focals_buddy/app/src/main/java/com/openfocals/services/screenmirror/ScreenFrameGrabber.java

You could replace this and provide your own method of picking what image to send. (Actual image sending is done by: /focals_buddy/app/src/main/java/com/openfocals/services/network/ScreenMirrorWSService.java )

The screensaver portion / making the image go away - you'd have to mess with the source code for the screen mirror app.