search

CasparCG / server

CasparCG Server is a Windows and Linux software used to play out professional graphics, audio and video to multiple outputs. It has been in 24/7 broadcast production since 2006. Ready-to-use downloads are available under the Releases tab https://casparcg.com.
GNU General Public License v3.0
902 stars 269 forks source link

Remove dependency on running x11 server #897

Open Julusian opened 6 years ago

Julusian commented 6 years ago

Related: #896 #782

It would be nice to be able to run CasparCG on linux without a running X11 server. this would allow us to run on Ubuntu server instead of Ubuntu desktop, and provides a proper setup for a headless machine.

This can be done by replacing SFML during the OpenGL context creation with EGL. This will need testing on windows, to ensure EGL is widely supported. (perhaps fallback to SFML if it is not?) It should also be tested over ssh, as the EGL configurations presented are different to those available over a local shell.

CEF currently (tested with CEF 59) requires X11 to initilialise. We should either look into if it is possible to remove this dependency, or disable CEF is X11 is not available. The DIAG window and screen consumer also rely on X11. As a first iteration, these could remain unavailable, but it would be useful in future to be able to use the screen consumer via EGL.

I prototyped this a while back, and got it to run with the html module removed. It should make a good reference. https://github.com/julusian/casparcg-server/tree/headless-linux

dimitry-ishenko commented 6 years ago

IMHO the DIAG window should be separated out from the server just like the media scanner. The server can push diagnostic messages to a socket. Then you could even run the DIAG window on a different computer. Maybe this can even be done with OSC?

Full-screen screen consumer can write to a frame buffer and not depend on X11. I don't know if anybody uses windowed screen consumer.

I don't know much about CEF, but does this looks promising?

https://bitbucket.org/chromiumembedded/cef/issues/2296/linux-add-ozone-mus-support-as-an

mint-dewit commented 6 years ago

IMHO the DIAG window should be separated out from the server just like the media scanner.

Trivial, considering diag info is already sent out via osc. It's been on my list for a little while now, but I haven't had reason to prioritize yet.

I don't know if anybody uses windowed screen consumer.

Actually, one of my clients uses it extensively for remote monitoring via teamview. I don't think we'd want to compromise on functionality there.

provides a proper setup for a headless machine.

I've done this before with some crazy combo of CUDA drivers + virtualgl + Xvfb which was all a bit of hack, so I'm not against doing this. The move to EGL sounds good to me (let's not act as if SFML won't give issues with ssh). Once consumers can instantiate their own context, perhaps they could instantiate a context against the ssh session rather than the machine.

Julusian commented 6 years ago

IMHO the DIAG window should be separated out from the server just like the media scanner. Trivial, considering diag info is already sent out via osc.

Yep, that is something Ive wanted to see done for a while, but havent gotten around to properly looking at it.

Full-screen screen consumer can write to a frame buffer and not depend on X11. I don't know if anybody uses windowed screen consumer.

By using EGL we can get a opengl frame buffer object for a display, meaning we can avoid any frames going via the cpu. I wouldnt want to remove the windowed screen consumer when we have X or are on windows, as it is useful to have especially when testing, but it being unavailable when X is not available should be fine.

I've done this before with some crazy combo of CUDA drivers + virtualgl + Xvfb

I have wondered about something like that, but it would be rather hacky and not an ideal setup. For now I am keeping an x server running (with openbox as the window manager) which works, but it could be nicer.

Once consumers can instantiate their own context, perhaps they could instantiate a context against the ssh session rather than the machine.

I don't think that would work. The server would only be able to do that over the ssh session it was launched over, and there wouldn't be enough bandwidth to send a channel uncompressed over the network.

Julusian commented 6 years ago

Potentially useful link: https://www.collabora.com/news-and-blog/blog/2017/12/22/cef-on-wayland/ https://chromium.googlesource.com/chromium/src/+/lkcr/docs/ozone_overview.md

deadbeef84 commented 4 years ago

It seems this might be possible now: https://bitbucket.org/chromiumembedded/cef/issues/2349/headless-mode-without-x

deadbeef84 commented 4 years ago

I played around with this a bit on a branch to try make it run in docker: https://github.com/deadbeef84/server/tree/headless (sorry, code is a bit of a mess as I just hacked it quickly)

It now starts in docker without x11, CEF logs an error on init [0121/163510.360222:ERROR:browser_main_loop.cc(1537)] Unable to open X display., and it crashes when trying to add an html producer:

[2020-01-21 16:35:10.275] [info]    ############################################################################
[2020-01-21 16:35:10.275] [info]    CasparCG Server is distributed by the Swedish Broadcasting Corporation (SVT)
[2020-01-21 16:35:10.275] [info]    under the GNU General Public License GPLv3 or higher.
[2020-01-21 16:35:10.275] [info]    Please see LICENSE.TXT for details.
[2020-01-21 16:35:10.275] [info]    http://www.casparcg.com/
[2020-01-21 16:35:10.275] [info]    ############################################################################
[2020-01-21 16:35:10.275] [info]    Starting CasparCG Video and Graphics Playout Server 2.3.0 N/A Dev
[2020-01-21 16:35:10.276] [info]    Initializing OpenGL Device.
[2020-01-21 16:35:10.303] [info]    Initialized OpenGL 4.6.0 NVIDIA 430.64 NVIDIA Corporation
[2020-01-21 16:35:10.307] [info]    Initialized ffmpeg module.
[2020-01-21 16:35:10.307] [info]    Initialized oal module.
[0121/163510.360222:ERROR:browser_main_loop.cc(1537)] Unable to open X display.
[2020-01-21 16:35:10.360] [info]    Initialized html module.
[2020-01-21 16:35:10.360] [info]    Initialized image module.
[2020-01-21 16:35:10.362] [info]    "/opt/casparcg/casparcg.config":
[2020-01-21 16:35:10.362] [info]    -----------------------------------------
[2020-01-21 16:35:10.362] [info]    <?xml version="1.0" encoding="utf-8"?>
[2020-01-21 16:35:10.362] [info]    <configuration>
[2020-01-21 16:35:10.362] [info]       <paths>
[2020-01-21 16:35:10.362] [info]          <media-path>media/</media-path>
[2020-01-21 16:35:10.362] [info]          <log-path>log/</log-path>
[2020-01-21 16:35:10.362] [info]          <data-path>data/</data-path>
[2020-01-21 16:35:10.362] [info]          <template-path>template/</template-path>
[2020-01-21 16:35:10.362] [info]       </paths>
[2020-01-21 16:35:10.362] [info]       <lock-clear-phrase>secret</lock-clear-phrase>
[2020-01-21 16:35:10.362] [info]       <channels>
[2020-01-21 16:35:10.362] [info]          <channel>
[2020-01-21 16:35:10.362] [info]             <video-mode>720p5000</video-mode>
[2020-01-21 16:35:10.362] [info]             <consumers>
[2020-01-21 16:35:10.362] [info]                <ffmpeg>
[2020-01-21 16:35:10.362] [info]                   <path>rtmp://10.0.1.224/WebRTCAppEE/M-3YNfnAtqjVNB</path>
[2020-01-21 16:35:10.362] [info]                   <args>-strict -2 -codec:a aac -b:a 128k -filter:a aresample=48000 -codec:v libx264 -g:v 50 -tune:v zerolatency -preset:v ultrafast -crf:v 25 -filter:v scale=nhd,fps=25,format=yuv420p -format flv</args>
[2020-01-21 16:35:10.362] [info]                </ffmpeg>
[2020-01-21 16:35:10.362] [info]             </consumers>
[2020-01-21 16:35:10.362] [info]          </channel>
[2020-01-21 16:35:10.362] [info]       </channels>
[2020-01-21 16:35:10.362] [info]       <controllers>
[2020-01-21 16:35:10.362] [info]          <tcp>
[2020-01-21 16:35:10.362] [info]             <port>5250</port>
[2020-01-21 16:35:10.362] [info]             <protocol>AMCP</protocol>
[2020-01-21 16:35:10.362] [info]          </tcp>
[2020-01-21 16:35:10.362] [info]       </controllers>
[2020-01-21 16:35:10.362] [info]       <amcp>
[2020-01-21 16:35:10.362] [info]          <media-server>
[2020-01-21 16:35:10.362] [info]             <host>localhost</host>
[2020-01-21 16:35:10.362] [info]             <port>8000</port>
[2020-01-21 16:35:10.362] [info]          </media-server>
[2020-01-21 16:35:10.362] [info]       </amcp>
[2020-01-21 16:35:10.362] [info]    </configuration>
[2020-01-21 16:35:10.362] [info]    -----------------------------------------
[2020-01-21 16:35:10.721] [info]    Initialized OpenGL Accelerated GPU Image Mixer for channel 1
[2020-01-21 16:35:10.721] [info]    video_channel[1|720p5000] Successfully Initialized.
[2020-01-21 16:35:10.722] [info]    ffmpeg[rtmp://10.0.1.224/WebRTCAppEE/M-3YNfnAtqjVNB] Initialized.
[2020-01-21 16:35:10.722] [info]    Initialized channels.
[2020-01-21 16:35:10.722] [info]    Initialized controllers.
[2020-01-21 16:35:10.723] [info]    Initialized osc.
[2020-01-21 16:35:10.733] [info]    [ffmpeg] [libx264 @ 0x7fa2a64e8100] using SAR=1/1
[2020-01-21 16:35:10.733] [info]    
[2020-01-21 16:35:10.736] [info]    [ffmpeg] [libx264 @ 0x7fa2a64e8100] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[2020-01-21 16:35:10.736] [info]    
[2020-01-21 16:35:10.737] [info]    [ffmpeg] [libx264 @ 0x7fa2a64e8100] profile Constrained Baseline, level 3.0
[2020-01-21 16:35:10.737] [info]    
CG 1 ADD 1 http://google.com 1
[2020-01-21 16:35:31.965] [info]    Received message from Console: CG 1 ADD 1 http://google.com 1\r\n
#501 CG ADD FAILED
[2020-01-21 16:35:31.970] [error]   Exception: /source/modules/html/html.cpp(334): Throw in function std::future<void> caspar::html::begin_invoke(const std::function<void()>&)
[2020-01-21 16:35:31.970] [error]   Dynamic exception type: boost::exception_detail::clone_impl<caspar::caspar_exception>
[2020-01-21 16:35:31.970] [error]   [caspar::tag_msg_info*] = [cef_executor] Could not post task
[2020-01-21 16:35:31.970] [error]   [caspar::tag_stacktrace_info*] =  0# caspar::stacktrace_info() in bin/casparcg
[2020-01-21 16:35:31.970] [error]    1# caspar::html::begin_invoke(std::function<void ()> const&) in bin/casparcg
[2020-01-21 16:35:31.970] [error]    2# caspar::html::invoke(std::function<void ()> const&) in bin/casparcg
[2020-01-21 16:35:31.970] [error]    3# caspar::html::html_producer::html_producer(caspar::spl::shared_ptr<caspar::core::frame_factory> const&, caspar::core::video_format_desc const&, std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const&) in bin/casparcg
...

@Julusian You seem to have some experience with CEF, and I did notice some API changes from 71->75 and I had to adjust the code slightly to make it work, do you have any clue why it's failing or how to debug it? It does seem to work if I remove the AppendSwitch("use-views") and AppendSwitchWithValue("ozone-platform", "headless").

This is new ground for me, and it's been a long time since I did any C++ development...

Julusian commented 4 years ago

@deadbeef84 yeah, cef always seems to make some small insignificant changes in their api.

From memory I think that CefInitialize has a return value which we are not handling, so I would guess that the crash is occuring because cef failed to initilize and trying to send it a task (loading the url) it than rejects.

I don't think you want to be using the headless platform. These docs say that headless will only output a png to file, and I guess that will upset cef.

Have you confirmed that those builds of cef have ozone enabled? I dont know where to check, but I had a feeling it wasnt enabled in the default builds.