UE 4.25 comaptibility - Githubissues

petergerten commented 4 years ago

Hi,

thanks you so much for you work on this projects. I saw that you mentioned 4.25 compatibility as "coming soon". Are there any news on this / are you still working on it?

Peter

robinzhx commented 4 years ago

Same question here. Not sure if there is any "hack" way to figure it out. I would definitely want to help develop on 4.25 compatibility

arthurbelair commented 4 years ago

Also interested to know if this is still planned for the future.

Thanks for the great work!

galeone commented 4 years ago

Hi @adamrehn do you have any update about how to integrate pixel streaming with ue 4.25/4.26?

lukehb commented 4 years ago

Hi all, just letting you know we (TensorWorks - Adam's company) are working on open source Pixel Streaming for 4.25 - it is coming! Sorry for the delays, we've been focussed on other client work.

galeone commented 4 years ago

That's amazing @lukehb ! Keep us posted :)

adamrehn commented 4 years ago

Apologies for the delay in responding to this issue, I've been drowning in work over the past few months and I'm slowly chipping away at my GitHub backlog whenever I get the chance.

As @lukehb said, we're hard at work getting 4.25 (and by extension, 4.26) support implemented and released to the community. Epic made extensive architectural changes to Pixel Streaming in both 4.24 and 4.25, so porting our existing work over from 4.23 has proven to be quite the undertaking, on top of which we've been dividing our time between our open source work and projects for consulting clients. At this point we've pretty much decided to skip 4.24 entirely so we can focus all of our attention on 4.25, since that's the first Engine version to support offscreen rendering with Vulkan. Fortunately the architecture of Pixel Streaming appears to have remained relatively unchanged in 4.26 (at least as of the most recent preview version), so porting our 4.25 changes to 4.26 once they're finished should hopefully be nice and straightforward.

ghost commented 4 years ago

Hi @adamrehn and thanks a lot for your work. Having offscreen rendering with Vulkan in 4.25 is awesome, as this might enable containerized ray tracing pixelstreaming goodness if UE decides to extend the support beyond dx12. On this regard I have a simple question, does ue4-docker support the build of ue4 images for Vulkan renderer?

adamrehn commented 4 years ago

@tibzuru yes, ue4-docker supports Vulkan, see this comment for details: https://github.com/adamrehn/ue4-docker/issues/76#issuecomment-615577529

adamrehn commented 3 years ago

@petergerten @robinzhx @arthurbelair @galeone @tibzuru I'm happy to report that the code for the 4.25 version of Pixel Streaming for Linux is now public and can be found here: https://github.com/ImmortalEmperor/UnrealEngine. I've also updated the article on my website to reflect the release, and we'll be publishing a blog post soon with details of the long journey to getting everything ported over to Vulkan.

luc122c commented 3 years ago

Hi @adamrehn , this is great news! Thanks so much for your hard work! In the article on your site, you have git link as https://github.com/adamrehn/ImmortalEmperor.git, should it be https://github.com/ImmortalEmperor/UnrealEngine as above?

avg07 commented 3 years ago

@luc122c, I used this configuration:

ue4-docker build \
  custom:4.25-pixelstreaming \
  -repo=https://github.com/ImmortalEmperor/UnrealEngine.git \
  -branch=4.25-pixelstreaming
  --cuda=10.2

luc122c commented 3 years ago

Mine appears to be stuck compiling the same files too: [builder 12/17] RUN ./Engine/Build/BatchFiles/RunUAT.sh BuildGraph -target="Make Installed Build Linux" -script=Engine/Build/InstalledEngineBuild 22188.1s

avg07 commented 3 years ago

Hi, @adamrehn. Why the build process takes so long? I ran:

ue4-docker build \
  custom:4.25-pixelstreaming \
  -repo=https://github.com/ImmortalEmperor/UnrealEngine.git \
  -branch=4.25-pixelstreaming
  --cuda=10.2

and the process has been going on for 2 days. The process is now at this stage:

    LogShaderCompilers: Display: Worker (3/5): shaders left to compile 16578
    LogShaderCompilers: Display: Worker (1/5): shaders left to compile 16568
    LogShaderCompilers: Display: Worker (2/5): shaders left to compile 16558
    LogShaderCompilers: Display: Worker (3/5): shaders left to compile 16548
    LogShaderCompilers: Display: Worker (4/5): shaders left to compile 16538

Perhaps I needed to add some build parameters? I need a full-image to build a ue-project with pixel-streaming.

adamrehn commented 3 years ago

@luc122c thanks for pointing out the typo in the article, I must've copy-pasted over the wrong segment of the URL. I've just fixed that now.

@luc122c @avg07 that's strange that it's taking so long, I'll run a build myself on Monday and see if I can reproduce the issue.

luc122c commented 3 years ago

Well, mine finally got built. It took just under 15 hours to complete ue4-minimal on a trashcan Mac Pro. Will try them out on Monday. Screenshot 2021-01-31 at 11 57 10 Screenshot 2021-01-31 at 11 56 50

avg07 commented 3 years ago

Hi @adamrehn. I have a problem with starting pixel-streaming on 4.25. When I using line in docker-build RUN ln -s /usr/lib/x86_64-linux-gnu/libnvidia-encode.so /home/ue4/project/ProjectName/Binaries/Linux/libnvidia-encode.so, I get an error:

[2021.01.31-12.07.44:433][  0]LogInit: Texture streaming: Enabled
[2021.01.31-12.07.44:445][  0]LogSlate: Updating window title bar state: overlay mode, drag disabled, window buttons hidden, title bar hidden
[2021.01.31-12.07.44:445][  0]LogInit: Display: Game Engine Initialized.
[2021.01.31-12.07.44:446][  0]LogCUDA: Initialising Cuda.
[2021.01.31-12.07.44:802][  0]LogCUDA: GPU Device 0 selected!
[2021.01.31-12.07.44:802][  0]LogAVEncoder: Available video encoders: None
[2021.01.31-12.07.44:802][  0]LogAVEncoder: Available audio encoders: , wmf(aac)
[2021.01.31-12.07.44:802][  0]PixelPlayer: Warning: Failed to create DXGI Manager and Device
[2021.01.31-12.07.44:802][  0]LogCore: Warning: dlopen failed: /home/ue4/project/MyProject4/Binaries/Linux/libnvidia-encode.so: cannot open shared object file: No such file or directory
[2021.01.31-12.07.44:802][  0]LogCudaVideoEncoder: Error: NVidia card found, but no NvEnc DLL installed.
[2021.01.31-12.07.44:802][  0]LogLinux: Warning: MessageBox: No compatible GPU found, or failed to load their respective encoder libraries: Pixel Streaming Plugin: No such audio device:
[2021.01.31-12.07.44:802][  0]Message dialog closed, result: Cancel, title: Pixel Streaming Plugin, text: No compatible GPU found, or failed to load their respective encoder libraries
[2021.01.31-12.07.44:802][  0]PixelStreamer: Error: No compatible GPU found, or failed to load their respective encoder libraries
[2021.01.31-12.07.44:803][  0]LogInit: Display: Starting Game.

But when I using line in docker-build RUN ln -s /usr/lib/x86_64-linux-gnu/libnvidia-encode.so.1 /home/ue4/project/ProjectName/Binaries/Linux/libnvidia-encode.so, I get an another error:

[2021.01.31-12.57.08:040][  0]LogInit: Initializing FReadOnlyCVARCache
[2021.01.31-12.57.08:040][  0]LogAudioMixer: Display: Audio Mixer Platform Settings:
[2021.01.31-12.57.08:040][  0]LogAudioMixer: Display:   Sample Rate:                                              48000
[2021.01.31-12.57.08:040][  0]LogAudioMixer: Display:   Callback Buffer Frame Size Requested: 10

#
# Fatal error in: ../../media/engine/adm_helpers.cc, line 38
# last system error: 0
# Check failed: 0 == adm->Init() (0 vs. -1)
# Failed to initialize the ADM.Signal 6 caught.
Malloc Size=65538 LargeMemoryPoolOffset=65554
24
[2021.01.31-12.57.08:040][  0]LogAudioMixer: Display:   Callback Buffer Frame Size To Use:        1024
[2021.01.31-12.57.08:040][  0]LogAudioMixer: Display:   Number of buffers to queue:                       2
[2021.01.31-12.57.08:040][  0]LogAudioMixer: Display:   Max Channels (voices):                            32
[2021.01.31-12.57.08:040][  0]LogAudioMixer: Display:   Number of Async Source Workers:           0
[2021.01.31-12.57.08:040][  0]LogAudio: Display: AudioDevice MaxSources: 32
[2021.01.31-12.57.08:041][  0]LogAudio: Display: Using built-in audio occlusion.
[2021.01.31-12.57.08:041][  0]LogAudioMixer: Display: Initializing audio mixer.
[2021.01.31-12.57.08:058][  0]LogAudioMixerSDL: Display: Initialized SDL using dsp platform API backend.
[2021.01.31-12.57.08:058][  0]LogAudioMixerSDL: Error: No such audio device
[2021.01.31-12.57.08:058][  0]LogAudioMixerSDL: Opening default audio device (device index -1)
[2021.01.31-12.57.08:058][  0]LogAudioMixerSDL: Error: No such audio device
[2021.01.31-12.57.08:059][  0]LogAudio: Warning: FAudioDevice::Init Failed!
[2021.01.31-12.57.08:060][  0]LogAudio: Display: Audio device could not be initialized. Please check the value for AudioDeviceModuleName and AudioMixerModuleName in [Platform]Engine.ini.
[2021.01.31-12.57.08:060][  0]LogAudio: Display: Audio device could not be initialized. Please check the value for AudioDeviceModuleName and AudioMixerModuleName in [Platform]Engine.ini.
[2021.01.31-12.57.08:060][  0]LogNetVersion: Set ProjectVersion to 1.0.0.0. Version Checksum will be recalculated on next use.
[2021.01.31-12.01.10:833][  0]LogInit: Texture streaming: Enabled
[2021.01.31-12.01.10:844][  0]LogSlate: Updating window title bar state: overlay mode, drag disabled, window buttons hidden, title bar hidden
[2021.01.31-12.01.10:845][  0]LogInit: Display: Game Engine Initialized.
[2021.01.31-12.01.10:846][  0]LogCUDA: Initialising Cuda.
[2021.01.31-12.01.11:179][  0]LogCUDA: GPU Device 0 selected!
[2021.01.31-12.01.11:179][  0]LogAVEncoder: Available video encoders: None
[2021.01.31-12.01.11:179][  0]LogAVEncoder: Available audio encoders: , wmf(aac)
[2021.01.31-12.01.11:179][  0]PixelPlayer: Warning: Failed to create DXGI Manager and Device
[2021.01.31-12.01.11:211][  0]LogCudaVideoEncoder: FCudaVideoEncoder initialization with 1920*1080, 60 FPS
[2021.01.31-12.01.11:251][  0]LogCudaVideoEncoder: Created texture 0x7fec37c66dc0 for CUDA
[2021.01.31-12.01.11:263][  0]LogCudaVideoEncoder: Created texture 0x7fec37c66ca0 for CUDA
[2021.01.31-12.01.11:276][  0]LogCudaVideoEncoder: Created texture 0x7fec37c66b80 for CUDA
[2021.01.31-12.01.11:287][  0]LogCudaVideoEncoder: CudaEnc initialised
[2021.01.31-12.01.11:289][  0]LogInit: Display: Starting Game.
[2021.01.31-12.01.11:289][  0]LogNet: Browse: /Game/TwinStickBP/Maps/TwinStickExampleMap?Name=Player
[2021.01.31-12.01.11:289][  0]LogLoad: LoadMap: /Game/TwinStickBP/Maps/TwinStickExampleMap?Name=Player
[2021.01.31-12.01.11:289][  0]LogWorld: BeginTearingDown for /Temp/Untitled_0
CommonUnixCrashHandler: Signal=6
[2021.01.31-12.01.11:339][  0]LogCore: === Critical error: ===
Unhandled Exception: SIGABRT: abort() called

[2021.01.31-12.01.11:339][  0]LogCore: Fatal error!

0x00007fec7cb52fb7 libc.so.6!gsignal(+0xc7)
0x00007fec7cb54921 libc.so.6!abort(+0x140)
0x000000000768d778 MyProject4!rtc::webrtc_checks_impl::FatalLog(char const*, int, char const*, rtc::webrtc_checks_impl::CheckArgType const*, ...)()
0x0000000007954de0 MyProject4!webrtc::adm_helpers::Init(webrtc::AudioDeviceModule*)()
0x000000000794ac2e MyProject4!cricket::WebRtcVoiceEngine::Init()()
0x000000000792d29d MyProject4!cricket::CompositeMediaEngine<cricket::WebRtcVoiceEngine, cricket::WebRtcVideoEngine>::Init()()
0x0000000007a17ce6 MyProject4!rtc::FunctorMessageHandler<bool, cricket::ChannelManager::Init()::$_2>::OnMessage(rtc::Message*)()
0x000000000769d4f5 MyProject4!rtc::MessageQueue::Dispatch(rtc::Message*)()
0x00000000076a70e8 MyProject4!rtc::Thread::ReceiveSendsFromThread(rtc::Thread const*)()
0x000000000769c7d6 MyProject4!rtc::MessageQueue::Get(rtc::Message*, int, bool)()
0x00000000076a6b88 MyProject4!rtc::Thread::Run()()
0x00000000076a6a7a MyProject4!rtc::Thread::PreRun(void*)()
0x00007fec7f55c6db libpthread.so.0!UnknownFunction(0x76da)
0x00007fec7cc3571f libc.so.6!clone(+0x3e)

[2021.01.31-12.01.11:343][  0]LogExit: Executing StaticShutdownAfterError
Engine crash handling finished; re-raising signal 6 for the default handler. Good bye.
Aborted (core dumped)

And in project build process I found:

PixelStreaming: Warning: Only D3D11/D3D12/Vulkan Dynamic RHI is supported. Detected Null
  PixelStreaming: Warning: Only D3D11/D3D12/Vulkan Dynamic RHI is supported. Detected Null
  SourceControl: Source control is disabled
  SourceControl: Source control is disabled
  SourceControl: Source control is disabled
  SourceControl: Source control is disabled
  SourceControl: Source control is disabled
  LogAndroidPermission: UAndroidPermissionCallbackProxy::GetInstance
  LogAudioCaptureCore: Display: No Audio Capture implementations found. Audio input will be silent.
  LogAudioCaptureCore: Display: No Audio Capture implementations found. Audio input will be silent.
  LogAudioCaptureCore: Display: No Audio Capture implementations found. Audio input will be silent.

avg07 commented 3 years ago

I solved the problem by adding a parameter to the docker-run command: -v ${XDG_RUNTIME_DIR}/pulse/native:${XDG_RUNTIME_DIR}/pulse/native. The problem was with the sound.

My docker-run: sudo docker run --rm --gpus=all -v ${XDG_RUNTIME_DIR}/pulse/native:${XDG_RUNTIME_DIR}/pulse/native --network=host demo-425:6 -PixelStreamingIP=127.0.0.1 -PixelStreamingPort=8888

K8s deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: pixel-streaming-deploy
spec:
  selector:
    matchLabels:
      app: pixelstreaming
  replicas: 1
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: pixelstreaming
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: kubernetes.io/hostname
                    operator: In
                    values:
                      - ue4-docker
      tolerations:
        - key: "GPU"
          operator: "Exists"
          effect: "NoSchedule"
      containers:
        - name: app
          image: "pixel-streaming/demo-425:6"
          args: ["-PixelStreamingIP=127.0.0.1", "-PixelStreamingPort=8888"]
          volumeMounts:
            - mountPath: /run/user/1000/pulse/native
              name: pulse-native
        - name: signallingwebserver
          image: "signallingwebserver:4.25-pixelstreaming"
          args: [ "--peerConnectionOptions", '{"iceServers":[{"urls":["stun:stun.l.google.com:19302"]}]}' ]
          ports:
            - name: entrypoint
              containerPort: 80
      volumes:
        - name: pulse-native
          hostPath:
            path: /run/user/1000/pulse/native
      imagePullSecrets:
        - name: regcred

adamrehn commented 3 years ago

@luc122c @avg07 apologies for taking longer than expected to get back to this, this week has been pretty crazy for me. I attempted to reproduce the lengthy build time issue by running the following command:

ue4-docker build custom:4.25-pixelstreaming -repo=https://github.com/ImmortalEmperor/UnrealEngine.git -branch=4.25-pixelstreaming --cuda=10.2 --no-engine

I left it to run overnight last night, and this morning I came back to find that the images had all built successfully, with the relevant output lines reporting build times reading as follows:

[ue4-docker build] Built image "adamrehn/ue4-minimal:4.25-pixelstreaming-cudagl10.2" in 2 hours, 32 minutes and 10.37 seconds

...

[ue4-docker build] Built image "adamrehn/ue4-full:4.25-pixelstreaming-cudagl10.2" in 3 minutes and 32.32 seconds

...

[ue4-docker build] Total execution time: 3 hours, 11 minutes and 0.62 seconds

Granted, my work machine is quite powerful (Ryzen 9 3950X CPU, 64GiB of RAM) and I skipped building the ue4-engine image, but it definitely wasn't anything like 2 whole days, and certainly not 15 hours for the ue4-minimal image. I'm genuinely a bit stumped as to what caused those lengthy build times that you guys observed (barring perhaps weaker hardware and the use of Docker Desktop under Windows or macOS to perform the build, but even then I wouldn't expect it to take that long.) I've updated the article to use the exact same command that worked nicely for me, so hopefully it'll work nicely for other people.

@avg07 thanks for reporting your issues with both the symlink location and PulseAudio, I'll investigate further to see if there are any tweaks that need to be made in how we're building WebRTC so that audio still works without needing the bind-mount from the host system.

lloves commented 3 years ago

I solved the problem by adding a parameter to the docker-run command: -v ${XDG_RUNTIME_DIR}/pulse/native:${XDG_RUNTIME_DIR}/pulse/native. The problem was with the sound.

My docker-run: sudo docker run --rm --gpus=all -v ${XDG_RUNTIME_DIR}/pulse/native:${XDG_RUNTIME_DIR}/pulse/native --network=host demo-425:6 -PixelStreamingIP=127.0.0.1 -PixelStreamingPort=8888

K8s deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: pixel-streaming-deploy
spec:
  selector:
    matchLabels:
      app: pixelstreaming
  replicas: 1
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: pixelstreaming
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: kubernetes.io/hostname
                    operator: In
                    values:
                      - ue4-docker
      tolerations:
        - key: "GPU"
          operator: "Exists"
          effect: "NoSchedule"
      containers:
        - name: app
          image: "pixel-streaming/demo-425:6"
          args: ["-PixelStreamingIP=127.0.0.1", "-PixelStreamingPort=8888"]
          volumeMounts:
            - mountPath: /run/user/1000/pulse/native
              name: pulse-native
        - name: signallingwebserver
          image: "signallingwebserver:4.25-pixelstreaming"
          args: [ "--peerConnectionOptions", '{"iceServers":[{"urls":["stun:stun.l.google.com:19302"]}]}' ]
          ports:
            - name: entrypoint
              containerPort: 80
      volumes:
        - name: pulse-native
          hostPath:
            path: /run/user/1000/pulse/native
      imagePullSecrets:
        - name: regcred

Thank you very much. Use your method solved the problem.

adamrehn commented 3 years ago

For those who had been dealing with the PulseAudio issue, there's now a proper fix for this that doesn't require disabling audio output or bind-mounting the PulseAudio socket from the host system: https://github.com/adamrehn/pixel-streaming-linux/issues/35

adamrehn / pixel-streaming-linux

UE 4.25 comaptibility #17