search

FrameOS / frameos

Operating system for single function smart frames
https://frameos.net/
Apache License 2.0
322 stars 8 forks source link

Tasklist #1

Open mariusandra opened 10 months ago

mariusandra commented 10 months ago

If you'd like to give a helping hand, pick any of the unfinished items and submit a PR. Don't ask for permission, everything is available and there are no issues to assign.

Phase 1: Python MVP

Completed by November 2023 - [x] Backend + frontend for control panel running - [x] SSH connection to frames - [x] Deploy and launch client software - [x] Launch as a daemon (via tmux for now) - [x] API connection back - [x] Streaming SSH logs to client - [x] UI to add frames - [x] Get live images back - [x] Reload button in FCP UI - [x] Show image as it's updating - [x] Sync frame updating state to control panel - [x] Sync button presses to control panel - [x] Add migrations - [x] Send logs sequentially - [x] Configure image via UI - [x] Kiosk mode URL - [x] Create new frame installation instructions (from rpi) - [x] Add apps - [x] Draw stuff on images - [x] OpenAI DallE2 - [x] Rotate images - [x] Scaling mode as part of the frame, not app - [x] App for scaling with cover modes - [x] Edit via a diagram UI - [x] Require user/pass to see interface - [x] Close a tab, fix bugs with tabs - [x] Handle saving modified apps - [x] Virtualize logs (list gets too big) - [x] Settings page with credential storage (HA, OpenAI, Github) - [x] Save scenes (one or many) as templates - [x] Template .zip export - [x] Template .zip import - [x] Github template repositories - [x] Support making screenshots of web pages (on 64bit devices only) - [x] Apps/devices must preselect the settings/secrets they are using, only those are copied over - [x] SSH keys - [x] Python and JSON errors inline in editor (red if file is obviously wrong) - [x] Editing text in app config jumps to end - [x] Document the coding environment and the available apps - [x] Get CPU, temp and other metrics from the frames - [x] Add Sentry support - [x] Better login screen - [x] If/else branches in diagram editor - [x] Horizontal/vertical split (grid mode)

By the end of August 2013, the MVP was essentially complete. I could deploy custom code from a library of building blocks over SSH onto a raspberry pi.

Phase 2: Nim rewrite

Can we 10x the efficiency? This might completely change the way apps are built, so best to explore it now instead of polishing more.

Completed January 2024 - [x] Explore migrating from PIL to [pixie-python](https://github.com/treeform/pixie-python) make sense? We'd get great font rendering for free, minus emoji support sadly. --> Might be great, but putting energy into the next point instead. - [x] Experiment if it makes any sense to rewrite the FrameOS on-device part in a compiled language. Conclusion: yes it does. We're doing real time graphics on low power devices. A compiled language brings 10x improvements in speed and/or performance. Migrating some parts to a compiled language is inevitable. After looking around a lot, nim seems like the best choice. People [praise it](https://news.ycombinator.com/item?id=36955806). It's friendlier than rust/zig/go (_"If you imagine native-compiled type-annotated Python where nearly 100% of your code is business logic with no cruft, you're getting close to the Nim experience."_), it's easy to pick up from examples, fast, it's [used for games](https://github.com/jmgomez/NimForUE) (it's fast), and comes with an [amazing image handling library](https://github.com/treeform/pixie) and other goodies. None of the other languages hit this sweet spot, so let's go. - [x] Rewrite FrameOS in nim. - [x] Proof of concept in the `nim` branch - [x] Deploy the nim code onto a raspberry with on-device compilation - [x] Caching for on device compilation - [x] Support for apps and the execution context - [x] Support numbers, color and other types of fields - [x] Support template strings in config -> inline code nodes for vars - [x] Support edited custom apps - [x] Convert to async server with schedules, etc - [x] Scheduler - [x] Inky impression display support? No C bindings? Only python available. Will call python code for now. - [x] Rotated display support - [x] System of hardware specific drivers, and devices that group them - [x] Port over the main apps - [x] Unsplash - [x] Text - [x] Code - [x] Clock - [x] Download URL - [x] FrameOS Gallery - [x] Color - [x] Gradient - [x] HA Sensor - [x] If - [x] Split - [x] Resize - [x] Rotate - [x] OpenAI - [x] Break if rendering - [x] Log batching in a separate thread - [x] Rendering in a separate thread - [x] HDMI support (double buffering, frame info, different color modes) - [x] Sanitize app fields - [x] Display on/off controls - [x] Skip most logs in high fps mode - [x] Metrics logger - [x] Waveshare eink support (using C libraries!) - [x] Touch and click handlers - [x] Ignore render when rendering, run all events after - [x] 60fps mode (max 2fps with Python now) -> got 60fps+ (more like 100) on a rpi5 with a 720p display (pikvm). So it's possible, even with simple "software rendering". Once we have the basics covered, I can investigate something like `boxy` to increase performance even more. - [x] Stability and hours of use (been running an inky impression + rpi zero w 1 and a hyperpixel round + zero w 2 at 24fps for days now without downtime) - [x] Show errors when editing nim apps in the interface - [x] Update docs https://github.com/FrameOS/frameos-docs/pull/1 - [x] Update the Dockerfile - [x] Merge the PR: https://github.com/FrameOS/frameos/pull/5 - [x] [Add B/W and B/W/R waveshare drivers](https://github.com/FrameOS/frameos/pull/28) - [x] [Full screen mode in web server](https://github.com/FrameOS/frameos/pull/34)

Read the blog post or the HN thread.

Phase 3: Work continues

January 2024. Improve driver support, stability and other fixes. - [x] Investigate timeouts hanging the logger - [x] [Delete old build folders on deploy](https://github.com/FrameOS/frameos/pull/33) - [x] Any way to lock/secure dependencies - [x] Global last image [lock](https://discord.com/channels/371759389889003530/371759389889003532/1192237215145803876) - [x] Cache deps in docker - [x] Show frame metric charts - [x] [Memory hole](https://github.com/FrameOS/frameos/pull/45)?? Memory is _not_ freed when requesting the image over http. - [x] Add [dithering](https://github.com/FrameOS/frameos/pull/44) for 2-color waveshare devices - [x] Add dithering [for the rest](https://github.com/FrameOS/frameos/pull/47) - [x] Add waveshare drivers - [x] Black/White - [x] 4-Grayscale - [x] 7-color - [x] 4-color Black/White/Red/Yellow - [x] Black/White/Yellow - [x] Black/White/Red - [x] Compile lgpio from scratch if no package - [x] Swap from string node ID-s in compiled [scenes to integers](https://github.com/FrameOS/frameos/pull/51) (should speed it up a bit and reduce memory usage) - [x] [GPIO button handlers](https://github.com/FrameOS/frameos/pull/52) - [x] Outgoing event nodes - [x] Disable swap (saves SD card from death if we have a memory leak) --> revert? messes up compilation on W2 - [x] [Faster compilation](https://github.com/FrameOS/frameos/pull/56) (was: local buildchains. Decided against it for now) - [x] Inky impressions native driver. Went with [faster python redeploys](https://github.com/FrameOS/frameos/pull/57) instead. The driver itself works fine as is, so not replacing today. - [x] Logger crashes again, and takes its thread to 100%. Let's try again... - [x] Debug toggle - [x] Double check all the `gcsafe` hacks - [x] General polish
February to April 2024. Multiple scenes, apps, regression tests and other completed tasks. - [x] App: [QR code](https://github.com/FrameOS/frameos/pull/58) - [x] App: GitHub star count - [x] App: GPT text - [x] [Scene state](https://github.com/FrameOS/frameos/pull/60) - [x] Scene state updates and persistence - [x] Scene state updates with built in web server - [x] [Auth for frame API access](https://github.com/FrameOS/frameos/pull/61) - [x] Don't drag from within textfields - [x] ~Sometimes 404 error when opening frame and clicking back~ only during rebuilds in dev - [x] [Multiple scenes](https://github.com/FrameOS/frameos/pull/63) - [x] Move parts of frame config to scene config/state - [x] [Revamped scenes and templates](https://github.com/FrameOS/frameos/pull/64) - [x] New default repository, easier sharing of scenes via repositories - [x] Visual regression tests for apps and scenes - [x] Tests for scene generation - [x] Update docs and screenshots - [x] Update docs about building apps - [x] [Easier split tool](https://github.com/FrameOS/frameos/pull/75) - [x] Initial scene wizard when setting up a new frame - [x] Improve path dragging/removing in the editor
May to June 2024. Data nodes, universal caching. - [x] Persist last open scene for restarts - [x] Schedule a restart - [x] Faster frontend builds - [x] Code nodes with arguments - [x] Universal caching controls - [x] Async data nodes (download url) - [x] Images as data (?) - [x] New nodes by dragging to the diagram - [x] Drag to empty space -> new node - [x] [Data apps, universal caching, refactors all old apps, context menus](https://github.com/FrameOS/frameos/pull/93) - [x] Cursor jumps to the end of the text field - [x] Unsplash app - [x] Update all templates with data apps - [x] Control render/sleep durations programmatically - [x] [Clean up nim app code](https://github.com/FrameOS/frameos/pull/97) - [x] Fix inputs cache type

Next things:

All the other ideas

Here's a list of things that have come to mind. Not in order of priority.

Cases and devices

andiohn commented 6 months ago

Hey! I LOVE THIS PROJECT SO MUCH!

Good work man. I am very inspired by this.

Ideas:

  1. Simple http served frame for crappy devices (kobo aura HD etc) with really old browsers.
  2. "touch targets" for those crappy devices (so you can pretend to interact with Home Assistant toggles.

Thank you for this. I'm going to try to use it and try to contribute.