Tasklist - Githubissues

If you'd like to give a helping hand, pick any of the unfinished items and submit a PR. Don't ask for permission, everything is available and there are no issues to assign.

Old archived tasks

July to November 2023. Python proof of concept

- [x] Backend + frontend for control panel running - [x] SSH connection to frames - [x] Deploy and launch client software - [x] Launch as a daemon (via tmux for now) - [x] API connection back - [x] Streaming SSH logs to client - [x] UI to add frames - [x] Get live images back - [x] Reload button in FCP UI - [x] Show image as it's updating - [x] Sync frame updating state to control panel - [x] Sync button presses to control panel - [x] Add migrations - [x] Send logs sequentially - [x] Configure image via UI - [x] Kiosk mode URL - [x] Create new frame installation instructions (from rpi) - [x] Add apps - [x] Draw stuff on images - [x] OpenAI DallE2 - [x] Rotate images - [x] Scaling mode as part of the frame, not app - [x] App for scaling with cover modes - [x] Edit via a diagram UI - [x] Require user/pass to see interface - [x] Close a tab, fix bugs with tabs - [x] Handle saving modified apps - [x] Virtualize logs (list gets too big) - [x] Settings page with credential storage (HA, OpenAI, Github) - [x] Save scenes (one or many) as templates - [x] Template .zip export - [x] Template .zip import - [x] Github template repositories - [x] Support making screenshots of web pages (on 64bit devices only) - [x] Apps/devices must preselect the settings/secrets they are using, only those are copied over - [x] SSH keys - [x] Python and JSON errors inline in editor (red if file is obviously wrong) - [x] Editing text in app config jumps to end - [x] Document the coding environment and the available apps - [x] Get CPU, temp and other metrics from the frames - [x] Add Sentry support - [x] Better login screen - [x] If/else branches in diagram editor - [x] Horizontal/vertical split (grid mode)

December to January 2024. Nim rewrite. Blog post. HN thread

- [x] Explore migrating from PIL to [pixie-python](https://github.com/treeform/pixie-python) make sense? We'd get great font rendering for free, minus emoji support sadly. --> Might be great, but putting energy into the next point instead. - [x] Experiment if it makes any sense to rewrite the FrameOS on-device part in a compiled language. Conclusion: yes it does. We're doing real time graphics on low power devices. A compiled language brings 10x improvements in speed and/or performance. Migrating some parts to a compiled language is inevitable. After looking around a lot, nim seems like the best choice. People [praise it](https://news.ycombinator.com/item?id=36955806). It's friendlier than rust/zig/go (_"If you imagine native-compiled type-annotated Python where nearly 100% of your code is business logic with no cruft, you're getting close to the Nim experience."_), it's easy to pick up from examples, fast, it's [used for games](https://github.com/jmgomez/NimForUE) (it's fast), and comes with an [amazing image handling library](https://github.com/treeform/pixie) and other goodies. None of the other languages hit this sweet spot, so let's go. - [x] Rewrite FrameOS in nim. - [x] Proof of concept in the `nim` branch - [x] Deploy the nim code onto a raspberry with on-device compilation - [x] Caching for on device compilation - [x] Support for apps and the execution context - [x] Support numbers, color and other types of fields - [x] Support template strings in config -> inline code nodes for vars - [x] Support edited custom apps - [x] Convert to async server with schedules, etc - [x] Scheduler - [x] Inky impression display support? No C bindings? Only python available. Will call python code for now. - [x] Rotated display support - [x] System of hardware specific drivers, and devices that group them - [x] Port over the main apps - [x] Unsplash - [x] Text - [x] Code - [x] Clock - [x] Download URL - [x] FrameOS Gallery - [x] Color - [x] Gradient - [x] HA Sensor - [x] If - [x] Split - [x] Resize - [x] Rotate - [x] OpenAI - [x] Break if rendering - [x] Log batching in a separate thread - [x] Rendering in a separate thread - [x] HDMI support (double buffering, frame info, different color modes) - [x] Sanitize app fields - [x] Display on/off controls - [x] Skip most logs in high fps mode - [x] Metrics logger - [x] Waveshare eink support (using C libraries!) - [x] Touch and click handlers - [x] Ignore render when rendering, run all events after - [x] 60fps mode (max 2fps with Python now) -> got 60fps+ (more like 100) on a rpi5 with a 720p display (pikvm). So it's possible, even with simple "software rendering". Once we have the basics covered, I can investigate something like `boxy` to increase performance even more. - [x] Stability and hours of use (been running an inky impression + rpi zero w 1 and a hyperpixel round + zero w 2 at 24fps for days now without downtime) - [x] Show errors when editing nim apps in the interface - [x] Update docs https://github.com/FrameOS/frameos-docs/pull/1 - [x] Update the Dockerfile - [x] Merge the PR: https://github.com/FrameOS/frameos/pull/5 - [x] [Add B/W and B/W/R waveshare drivers](https://github.com/FrameOS/frameos/pull/28) - [x] [Full screen mode in web server](https://github.com/FrameOS/frameos/pull/34) - [x] Investigate timeouts hanging the logger - [x] [Delete old build folders on deploy](https://github.com/FrameOS/frameos/pull/33) - [x] Any way to lock/secure dependencies - [x] Global last image [lock](https://discord.com/channels/371759389889003530/371759389889003532/1192237215145803876) - [x] Cache deps in docker - [x] Show frame metric charts - [x] [Memory hole](https://github.com/FrameOS/frameos/pull/45)?? Memory is _not_ freed when requesting the image over http. - [x] Add [dithering](https://github.com/FrameOS/frameos/pull/44) for 2-color waveshare devices - [x] Add dithering [for the rest](https://github.com/FrameOS/frameos/pull/47) - [x] Add waveshare drivers - [x] Black/White - [x] 4-Grayscale - [x] 7-color - [x] 4-color Black/White/Red/Yellow - [x] Black/White/Yellow - [x] Black/White/Red - [x] Compile lgpio from scratch if no package - [x] Swap from string node ID-s in compiled [scenes to integers](https://github.com/FrameOS/frameos/pull/51) (should speed it up a bit and reduce memory usage) - [x] [GPIO button handlers](https://github.com/FrameOS/frameos/pull/52) - [x] Outgoing event nodes - [x] Disable swap (saves SD card from death if we have a memory leak) --> revert? messes up compilation on W2 - [x] [Faster compilation](https://github.com/FrameOS/frameos/pull/56) (was: local buildchains. Decided against it for now) - [x] Inky impressions native driver. Went with [faster python redeploys](https://github.com/FrameOS/frameos/pull/57) instead. The driver itself works fine as is, so not replacing today. - [x] Logger crashes again, and takes its thread to 100%. Let's try again... - [x] Debug toggle - [x] Double check all the `gcsafe` hacks - [x] General polish

February to April 2024. Multiple scenes, apps, regression tests and other completed tasks.

- [x] App: [QR code](https://github.com/FrameOS/frameos/pull/58) - [x] App: GitHub star count - [x] App: GPT text - [x] [Scene state](https://github.com/FrameOS/frameos/pull/60) - [x] Scene state updates and persistence - [x] Scene state updates with built in web server - [x] [Auth for frame API access](https://github.com/FrameOS/frameos/pull/61) - [x] Don't drag from within textfields - [x] ~Sometimes 404 error when opening frame and clicking back~ only during rebuilds in dev - [x] [Multiple scenes](https://github.com/FrameOS/frameos/pull/63) - [x] Move parts of frame config to scene config/state - [x] [Revamped scenes and templates](https://github.com/FrameOS/frameos/pull/64) - [x] New default repository, easier sharing of scenes via repositories - [x] Visual regression tests for apps and scenes - [x] Tests for scene generation - [x] Update docs and screenshots - [x] Update docs about building apps - [x] [Easier split tool](https://github.com/FrameOS/frameos/pull/75) - [x] Initial scene wizard when setting up a new frame - [x] Improve path dragging/removing in the editor

May to July 2024. Data nodes, state nodes, universal caching, agenda app.

- [x] Persist last open scene for restarts - [x] Schedule a restart - [x] Faster frontend builds - [x] Code nodes with arguments - [x] Universal caching controls - [x] Async data nodes (download url) - [x] Images as data (?) - [x] New nodes by dragging to the diagram - [x] Drag to empty space -> new node - [x] [Data apps, universal caching, refactors all old apps, context menus](https://github.com/FrameOS/frameos/pull/93) - [x] Cursor jumps to the end of the text field - [x] Unsplash app - [x] Update all templates with data apps - [x] Control render/sleep durations programmatically - [x] [Clean up nim app code](https://github.com/FrameOS/frameos/pull/97) - [x] Fix inputs cache type - [x] Logger leaks memory (stopped buffering, got better, added workaround: nightly restarts) - [x] [iCal support](https://github.com/FrameOS/frameos/pull/79) - [x] Simple timezone support - [x] Save OpenAI images to disk - [x] Edit apps with multiple files - [x] State nodes - [x] Document building scenes - [x] Build rich text app - [x] Build agenda app - [x] Set system timezone - [x] Assets browser - [x] XKCD scene - [x] Push edge when adding node

TODO next

[x] Show scene.nim errors
[ ] Fix logger bugs
[ ] Upload images
[ ] Control URL: don't render immediately when swapping scenes
[ ] Store diagram/edit state in URL
[ ] Drag to select / select multiple
[ ] Copy/paste nodes
[ ] Autocomplete for nim code
[ ] Monaco for code nodes
[ ] Clean up edges that are not connected

Apps to build next

[ ] Calendar
[ ] Graphs and charts
[ ] Google Photos
[ ] RTSP video stream
[ ] Image transitions / screensavers (e.g time based smooth zoom)

New features

[ ] Cross compilation
[ ] Act as wifi hotspot for initial setup
[ ] Custom event types
[ ] Apps and versioning
[ ] Scenes in scenes & special "Layout" scene
[ ] Visual, grid, layout, placement, etc editor
[ ] Errors and exceptions in the diagram
[ ] Scheduler nodes

All the other ideas

Here's a list of things that have come to mind. Not in order of priority.

[ ] Video intro
[ ] Release 1.0 (use calver, so release 24.07 or so)
[ ] Automatically update timezones
[ ] Upload/sync assets
[ ] Tests for server
[ ] Tests for metrics
[ ] Tests for drivers
[ ] Community forums and device guides
[ ] CI for all supported raspberry OS versions
[ ] Tests for scene runners
[ ] Backups
[ ] Config for drivers (choose which pins are buttons, etc)
[ ] Kindle support
[ ] ESP32 support
[ ] JS target for live editing
[ ] Font with emoji support
[ ] Better sharing/import/export of apps
[ ] Verify support for all other inky frames
[ ] Unified background data fetching to not slow down real time mode
[ ] Throttle if CPU is hot
[ ] Slow down or turn off rendering if display is off
[ ] HTTPS access between frames and controllers
[ ] 404 on frame that is not there
[ ] SSH terminal
[ ] Multiple SSH keys
[ ] Package as a home assistant or Hacs addon
[ ] Events (e.g. GPIO buttons) as home assistant events
[ ] Boot splash screen for hdmi devices
[ ] UI polish, design system work, etc
[ ] Hyperpixel native driver (screen turn on/off is via python now)
[ ] Investigate GPU rendering via boxy
[ ] Pallette support for type="color" node config on dithered displays
[ ] New image data app: % option
[ ] Resize image data app % option
[ ] Resize image data app: center properly if source is smaller than target

Cases and devices

[ ] Create a modular, easy, and semi-compatible 3d printed case system for all these different frames and raspberries.
[ ] Update the device guides with a lot more info and images
[ ] Clean system for adding frames/drivers/capabilities

FrameOS / frameos

Tasklist #1