timeline ui - Githubissues

mediar-ai / screenpipe

24/7 local AI screen & mic recording. Build AI apps that have the full context. Works with Ollama. Alternative to Rewind.ai. Open. Secure. You own your data. Rust.

https://screenpi.pe

MIT License

7.04k stars 368 forks source link

timeline ui #343

Open louis030195 opened 2 weeks ago

louis030195 commented 2 weeks ago

i have been reticent for a while on this feature because screenpipe is not intented to be a copy of rewind or things like this and more focused on infrastructure / devtools

but this is more intuitive

https://screenmemory.app/

https://www.rewind.ai/

i guess all that is required is:

[x] merge mp4 endpoint
[ ] plumbing with audio
[ ] global shortcut
[ ] (later) select part of your day to prompt AI
[ ] maybe: annotate times with LLM/pipes
[ ] eventually all pipes could be somehow integrated with this e.g. "enrich my CRM pipe" would show somewhere i had conversation with customer john and click bring to salesforce or idk
[ ] ?

linear[bot] commented 2 weeks ago

MED-107 timeline ui

louis030195 commented 2 weeks ago

https://github.com/tauri-apps/tauri/discussions/4810

craftyshaun commented 2 weeks ago

but this is more intuitive

My 2c here is that this adds the ability to 'sense check' the LLM response, which is excellent. I'm using Reclaim and saw this as a near drop-in. I was just about to purchase it, but then I noticed that the timeline was missing.

Understanding the reclaim.ai has also pivoted to the pendant may create a market gap as well.

louis030195 commented 1 week ago

bounty for a small part of this, e.g. just showing a scrollable timeline above current app i'm using even if it's full screen

definition of done:

suggest some designs (excalidraw, figma, whatever) and Ill validate
when i do specific shortcut from any app, it shows a scrollable timeline (can be fake data initially, i can do the plumbing with .mp4)
works even if i'm in a full screen app
works on macos and windows at least

Glavin001 commented 6 days ago

Timeline view with Rewind was one of my favourite and most used features. Would be a must have for me to switch from Rewind to screenpipe completely. Made it super quick to look back in time to see my screen, copy text (you already have OCR ✅ ), etc.

louis030195 commented 5 days ago

biggest blocker for this is how to show window on top of full screen app in tauri?

craftyshaun commented 5 days ago

how to show window on top of full screen app in tauri?

If you're looking to emulate the Rewind.ai app then this Is understandable.

That said if I was to make a user story, I'd say:

As a user, I want to view the mp4 history with the timestamp to align it with the AI responses. This will enable me to check the response and eyeball what's on the screen if the LLM provides a subpar result.

As for the UX I would not mind if it were another window with the same ratio as my screen yet smaller (or even resizable), and I could zoom in if need be. As long as I can 'change to that window' and focus on it while viewing, I would not mind.

Tauri, as your platform, will have pros and cons. This may be a time when speed of development is prioritised over 'the perfect UI'.

@Glavin001, does the above resinate with you?

louis030195 commented 5 days ago

an idea i had also but kinda different is to allow user to scroll horizontally the video in results here

louis030195 commented 5 days ago

but i think the main request for this timeline ui is to be able to search, fast, anywhere, while now it's more friction to get more precise results

so timeline to my understanding is more a low friction, quick result and the scrollable timeline feels quite natural compared to a complex search interface

louis030195 commented 5 days ago

also if we ship multimodal embedding we could do multimodal semantic search which could be different / interesting UX