WIP uni{fied,versal} capture library/module

Informatic commented 2 years ago

This is a work in progress branch with "new" synchronous capture backend interface, unified for both UI and Video layer capture, where all pixel format conversion, scaling and blending is extracted into a common "unicapture" module.

backends now expose a simple, unified synchronous interface for a specific layer capture (ie. we now have separate backends for video layers - dile_vt, vtcapture, and ui layers - gm and halgal)
unicapture module is responsible for backend initialization, frame keeping (vsync) and merging/blending/conversion of different layers.
converter is responsible for converting frames between colorspaces (via libyuv), intentionally used only in unicapture
service is a module that binds unicapture module with hyperion_client and exposes Luna bus management service
settings is a module responsible for converting settings_t structure to/from JSON objects and files (stored in config.json file)
main only handles initialization and startup of service

Notable changes:

Added support for UI capture when combined with dile_vt
Added UI/video backend autodetection (default)
Added support for automatic pausing of capture when the TV is going to suspend mode (QuickStart+)
Added "vsync": false option that will help with issues like #30 ("fake" vsync signal will be issued every 1/FPS seconds, instead of using video backend waiting method)
Unified support for novideo/nogui options in all backends
Fixed/improved FPS counting/limiting on all platforms
/status luna service endpoint returns current framerate and info about currently running capture modules
Disabling autostart will remove the init.d symlink properly now
General stability improvements
Code has been reformatted using clangd/clang-format

Known issues:

TODO: Backend/resolution change requires service kill and restart
Due to some luna endpoint changes current PicCap GUI does not fully work properly yet
libvt backend has not been ported to "v2" backend interface, and is effectively broken after merging this

Supersedes:

47
48 - added vsync flag
40 - fixed frame pace handling including optional vsync
21 - contains support for libgm & libhal_gal UI capture support

chbartsch commented 2 years ago

FYI: You can find my work in progress on the halgal/vtcapture side here.

If I can help with stuff or test things just let me know. ;)

TBSniller commented 2 years ago

@Informatic: Hope its okay for you, I have commited a change in your pull request. If the service is not running as root, but graphic capture is enabled the process crashes while initiating, because its not running as root. This is the case after very first PicCap start.

tuxuser commented 2 years ago

~~- Do we want to enable the -S (--no-service) switch again? https://github.com/webosbrew/hyperion-webos/blob/6025215d450c0aaf731f0f8d22872e5addbf9ce7/src/main.c#L46 https://github.com/webosbrew/hyperion-webos/blob/6025215d450c0aaf731f0f8d22872e5addbf9ce7/src/main.c#L90-L92~~

~~- -s/--save-conf argument still needed?~~ https://github.com/webosbrew/hyperion-webos/blob/6025215d450c0aaf731f0f8d22872e5addbf9ce7/src/main.c#L58 https://github.com/webosbrew/hyperion-webos/blob/6025215d450c0aaf731f0f8d22872e5addbf9ce7/src/main.c#L114-L116

Do not write framedumps to /tmp/ (or make it toggleable) https://github.com/webosbrew/hyperion-webos/blob/6025215d450c0aaf731f0f8d22872e5addbf9ce7/src/unicapture.c#L253-L260

Do not show statistics when backend-init failed, cause its a little bit confusing? Example:

[INFO unicapture_run      ] Framerate: 9.240378 FPS; timings - wait: 10721us, acquire: 3047us, convert: 40502us, process; 475us, send: 38036us, release: 2us
[INFO unicapture_run      ]         UI: pixfmt: 11; 320x180
[INFO unicapture_run      ]      VIDEO: pixfmt: 0; 0x0
[INFO unicapture_run      ] CONV    UI: pixfmt: 12; 320x180
[INFO unicapture_run      ] CONV VIDEO: pixfmt: 0; 0x0

Longer log snippet for context:

     0.082s [WARN capture_init        ] [DILE_VT] DISPLAY dump location failed, attempting SCALER...
     0.082s [ ERR capture_init        ] [DILE_VT] SetVideoFrameOutputDeviceDumpLocation failed!
     0.084s [ ERR unicapture_init_backend] libdile_vt_backend.so: init failure, code: -2
     0.084s [WARN unicapture_try_backends] try_backends: backend: libdile_vt_backend.so failed with code: -2
     0.084s [ ERR unicapture_try_backends] Try backends failed!
     0.089s [ DBG unicapture_run      ] Attempting UI capture init...
     0.089s [ DBG main                ] Going into main loop..
     0.089s [ DBG connection_loop     ] Starting connection loop
     0.089s [INFO unicapture_vsync_handler] vsync thread starting...
     0.089s [INFO unicapture_run      ] UI capture started
     0.089s [INFO connection_loop     ] Connecting hyperion-client..
     0.093s [INFO power_callback      ] Power status callback message: { "subscribed": true, "returnValue": true, "state": "Active" }
     0.175s [INFO unicapture_run      ] Buffer dumped to: /tmp/hyperion-webos-dump.000.data
     0.213s [INFO connection_loop     ] hyperion-client connected!
     2.478s [INFO power_callback      ] Power status callback message: { "returnValue": true, "state": "Active", "processing": "Request Screen Saver" }
     2.478s [INFO power_callback      ] Pausing due to power event...
     2.495s [INFO unicapture_run      ] Shutting down...
     2.495s [ DBG unicapture_run      ] Waiting for vsync thread to finish...
     2.533s [INFO unicapture_vsync_handler] vsync thread finished
     2.534s [ DBG unicapture_run      ] Terminating UI capture...
     2.534s [ DBG unicapture_run      ] Done!
     2.537s [INFO connection_loop     ] Ending connection loop
     2.537s [ DBG connection_loop     ] Connection loop exiting
     2.538s [INFO power_callback      ] Power status callback message: { "returnValue": true, "state": "Active", "processing": "Screen On" }
     2.538s [INFO power_callback      ] Power status callback message: { "returnValue": true, "state": "Active" }
     2.538s [INFO power_callback      ] Resuming after power pause...
     2.538s [ DBG connection_loop     ] Starting connection loop
     2.538s [INFO unicapture_vsync_handler] vsync thread starting...
     2.538s [ DBG unicapture_run      ] Attempting UI capture init...
     2.538s [INFO connection_loop     ] Connecting hyperion-client..
     2.538s [INFO unicapture_run      ] UI capture started
     2.557s [INFO connection_loop     ] hyperion-client connected!
     2.623s [INFO unicapture_run      ] Buffer dumped to: /tmp/hyperion-webos-dump.000.data
     6.149s [INFO unicapture_run      ] Buffer dumped to: /tmp/hyperion-webos-dump.001.data
     9.031s [INFO unicapture_run      ] Framerate: 9.240378 FPS; timings - wait: 10721us, acquire: 3047us, convert: 40502us, process; 475us, send: 38036us, release: 2us
     9.031s [INFO unicapture_run      ]         UI: pixfmt: 11; 320x180
     9.031s [INFO unicapture_run      ]      VIDEO: pixfmt: 0; 0x0
     9.031s [INFO unicapture_run      ] CONV    UI: pixfmt: 12; 320x180
     9.031s [INFO unicapture_run      ] CONV VIDEO: pixfmt: 0; 0x0

Update: point 1 and 2 make no sense anymore with the way unicapture works :) Update 2: Changes done, PR sent.

tuxuser commented 2 years ago

When following PRs are merged, unicapture could be merged too:

PR #79
PR #82

TBSniller commented 2 years ago

@Informatic @tuxuser @chbartsch after almost one year it's done. Thank you all! <3

webosbrew / hyperion-webos

WIP uni{fied,versal} capture library/module #43

47

48 - added `vsync` flag

40 - fixed frame pace handling including optional vsync

21 - contains support for libgm & libhal_gal UI capture support

webosbrew / hyperion-webos

WIP uni{fied,versal} capture library/module #43

47

48 - added vsync flag

40 - fixed frame pace handling including optional vsync

21 - contains support for libgm & libhal_gal UI capture support

48 - added `vsync` flag