yatt-ai / yattie

A tool to help testers test.

GNU General Public License v3.0

60 stars 10 forks source link

feat: Capture button presses on screen recording #67

Open dacoaster opened 1 year ago

dacoaster commented 1 year ago

Feature Request

Capture button presses and mouse clicks on screen recording and display the text at the bottom of the screen in the recording.

dacoaster commented 6 months ago

@devknight216

devknight216 commented 6 months ago

@dacoaster , can you explain this feature in detail, plz?

dacoaster commented 6 months ago

@devknight216 - Something that will show mouse clicks and keystrokes so it is easy for someone watching the video to follow along like

devknight216 commented 6 months ago

@dacoaster , just wanted to ask how to implement this feature since I face an obstacle. In my perspective, there were two apporoaches to do this:

First option was to embed the captured events array to the captured video using ffmpeg, but it takes long time to process all.
Second option was to make a textOverlay over the app when the keyboard presses so that the captured video contains the screen as well as the event text. But the problem here is that the textOverlay should be system-wide in case capturing the other screen or window apart from the yotti app. I tried to make another transparent screen with the text so that it appears on top of all screens, but it is not capturing as a video as there is no option to capture two sources at the same time.

Please let me know your thought regarding this problem.

dacoaster commented 6 months ago

@devknight216 how much slower does it make the processing? we already have issues with slow processing that need to be addressed so maybe you could look at this issue as well and solve both?

devknight216 commented 6 months ago

@dacoaster , it takes really long. I tested it with one text of timeframe 3:00~5:00 seconds during the video(40seconds). It takes 1min on my laptop to embed the text.

devknight216 commented 6 months ago

Let me find the option to decrease process time. Maybe reducing the resolution could be the ffmpeg setting optimization can be used, but not sure it will decrease the time tremendously.

dacoaster commented 6 months ago

How long does it take without the text embedding?

dacoaster commented 6 months ago

Resolution should be configurable via the config. We should change the default to lower. We should also check if other codecs are faster or if there is a way to make it multi threaded or anything like that

devknight216 commented 6 months ago

I didn't test the optimization process, just did text embedding. I will test the optimization process with different codes as well.

dacoaster commented 6 months ago

thank you!