search

voxeet / comms-sdk-unreal

Dolby.io Virtual Worlds plugin for Unreal Engine
http://api-references.dolby.io/comms-sdk-unreal/
MIT License
26 stars 7 forks source link

UE5 Server MAC build crash with DolbyIOModule error #415

Closed AntonIovbakDev closed 10 months ago

AntonIovbakDev commented 1 year ago

Describe the bug

After packaging UE5 Server on MAC crash with DolbyIOModule error

Expected Behavior

[Describe the expected behavior of the feature.]

Specifications

Screen Shot 2023-09-20 at 10 11 05 AM

djova-dolby commented 11 months ago

It looks like it is failing here to load the first library it tried to find. I found one recent release note that indicates a change to library path creation. Not sure if this is the issue though so I have some question:

The reason I ask is because I see there is another issue reported for 1.2.1 for Windows that seems similar about regarding game packaging, which I assume was working before, but could be wrong. If it did work before it is possible that something like this change will fix it and here are the artifacts for this attempt.

AntonIovbakDev commented 11 months ago

Hi guys!

  1. Packaging for MAC was working for 1.1.6, BUT the packaged game crashes on start. For the last one (1.2.1) packaging is not successful at all. We have to fix it due to fixed issues in 1.2.1.
  2. We still have a problem, when you share window (some app) and close this app during sharing - we have crash.
  3. Also, we've found a bug with 2DRHI texture. Looks like you are passing RHI texture ref and 2DTexture size, and in some cases, it causes UpdateTexture error and crash:

[2023.09.15-07.51.28:627][728]LogWindows: Error: Assertion failed: UpdateRegion.DestX + UpdateRegion.Width <= Texture->GetSizeX() [File:C:\UnrealEngine\Engine\Source\Runtime\RHI\Public\RHICommandList.h] [Line: 812] [2023.09.15-07.51.28:627][728]LogWindows: Error: UpdateTexture2D out of bounds on X. Texture: Texture2D_2147480439, 0, 480, 320

We've made fix and it is work:

[cid:82347bd2-2d39-44a9-a780-553b3f12a82c] Best regards, Engineering Lead Anton Iovbak

From: Dejan Jovasevic @.> Sent: Wednesday, October 4, 2023 8:38 PM To: DolbyIO/comms-sdk-unreal @.> Cc: Anton Iovbak @.>; Author @.> Subject: Re: [DolbyIO/comms-sdk-unreal] UE5 Server MAC build crash with DolbyIOModule error (Issue #415)

It looks like it is failing here to load the first library it tried to find. I found one recent release note that indicates a change to library path creation. Not sure if this is the issue though so I have some question:

— Reply to this email directly, view it on GitHubhttps://github.com/DolbyIO/comms-sdk-unreal/issues/415#issuecomment-1747357035, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCQROF45ISEE5BJTWTOSG5DX5WNJBAVCNFSM6AAAAAA47PXWKGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBXGM2TOMBTGU. You are receiving this because you authored the thread.Message ID: @.***>

djova-dolby commented 11 months ago

We've made fix and it is work: [cid:82347bd2-2d39-44a9-a780-553b3f12a82c]

what does this mean? What fix have you made and what was it for (packaging or the crashes with the texture); is it in the Unreal SDK? Could you share/submit it to some branch?

When was the last time you packaged a game and it didn't crash on mac? Or does it always crash?

AntonIovbakDev commented 11 months ago
  1. This fix into your plugin's file. DolbyIOVideoTexture.cpp, void FVideoTexture::Render() method.
  2. It is always crash with Dolby, even if it is packaged.

BR Anton

From: Dejan Jovasevic @.> Sent: Thursday, October 5, 2023 11:38 AM To: DolbyIO/comms-sdk-unreal @.> Cc: Anton Iovbak @.>; Author @.> Subject: Re: [DolbyIO/comms-sdk-unreal] UE5 Server MAC build crash with DolbyIOModule error (Issue #415)

We've made fix and it is work: [cid:82347bd2-2d39-44a9-a780-553b3f12a82c] what does this mean? What fix have you made and what was it for; is it in the Unreal SDK? Could you share/submit it to some branch?

When was the last time you packaged a game and it didn't crash on mac? Or does it always crash?

— Reply to this email directly, view it on GitHubhttps://github.com/DolbyIO/comms-sdk-unreal/issues/415#issuecomment-1748384864, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCQROFZGQIAGZZBMV6FZTKDX5ZWZXAVCNFSM6AAAAAA47PXWKGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBYGM4DIOBWGQ. You are receiving this because you authored the thread.Message ID: @.***>

djova-dolby commented 11 months ago

Ok so it just for rendering, not packaging. Since this project is completely open source can you upstream the fix so we can merge it?

And then to reiterate the previous question about plugin packaging; it is using the DolbyIOModule.cpp from CoSocietyUnreal/Plugins... folder but looking for the library inside CoSociety/Plugins.... is this how plugins are normally loaded or should it be in the same folder? Like could this be the bug that the DolbyIOModule.cpp code is not setting proper path or something?

djova-dolby commented 11 months ago

Can you try with the artifacts from this run? https://github.com/DolbyIO/comms-sdk-unreal/actions/runs/6429286687 Also what is UE5 Server on MAC is this like a different way of packaging of Unreal or something like this? Google has not been helpful here. @Krysed was able to package game on macOS from what I understand with UE5.

djova-dolby commented 11 months ago

@Krysed and I have been able to use the 1.2.2-beta.1 plugin from the latest artifacts and package the [internal repo]() application on x86/arm macOS platforms. We were able to run the resulting application on both platforms. So not really sure where the issue is on yours? Does this UE5 server concept change things, or is this irrelevant?

AntonIovbakDev commented 11 months ago

Hi!

Cannot open https://github.com/dolbyio-samples/comms-unreal-demo-internal/tree/main

BR Anton

From: Dejan Jovasevic @.> Sent: Monday, October 9, 2023 10:22 AM To: DolbyIO/comms-sdk-unreal @.> Cc: Anton Iovbak @.>; Author @.> Subject: Re: [DolbyIO/comms-sdk-unreal] UE5 Server MAC build crash with DolbyIOModule error (Issue #415)

@Krysedhttps://github.com/Krysed and I have been able to use the 1.2.2-beta.1 plugin from the latest artifacts and package the Unreal Demohttps://github.com/dolbyio-samples/comms-unreal-demo-internal/tree/main application on x86/arm macOS platforms. We were able to run the resulting application on both platforms. So not really sure where the issue is on yours? Does this UE5 server concept change things, or is this irrelevant?

— Reply to this email directly, view it on GitHubhttps://github.com/DolbyIO/comms-sdk-unreal/issues/415#issuecomment-1752466748, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCQROF5QPICQXMK2E2OUI3LX6OQ4DAVCNFSM6AAAAAA47PXWKGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJSGQ3DMNZUHA. You are receiving this because you authored the thread.Message ID: @.***>

djova-dolby commented 11 months ago

Sorry I added the wrong link. It should be [internal repo]

djova-dolby commented 11 months ago

Note that we were also able to package/run using the 1.2.1 version of the plugin on macOS, so I am not sure that 1.2.2 changes will affect you. Likely you will be running into the same issue you had before. So to re-iterate something that we may not be understanding; what does it exactly mean UE5 Server MAC build? Is this a different way to package the game that we need to try? Or does it mean something else? Like from what I understand you are still using the plugin as intended for client application, so clarifying this part could be the part we are missing.

AntonIovbakDev commented 11 months ago

Excuses, man) but still the same. Is it a public repo?

BR Anton

From: Dejan Jovasevic @.> Sent: Monday, October 9, 2023 1:31 PM To: DolbyIO/comms-sdk-unreal @.> Cc: Anton Iovbak @.>; Author @.> Subject: Re: [DolbyIO/comms-sdk-unreal] UE5 Server MAC build crash with DolbyIOModule error (Issue #415)

Sorry I added the wrong link. It should be https://github.com/dolbyio-samples/comms-unreal-demo

— Reply to this email directly, view it on GitHubhttps://github.com/DolbyIO/comms-sdk-unreal/issues/415#issuecomment-1752751939, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCQROF5VPYXQEUBEOAK2ACDX6PG7JAVCNFSM6AAAAAA47PXWKGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJSG42TCOJTHE. You are receiving this because you authored the thread.Message ID: @.***>

AntonIovbakDev commented 11 months ago

Ok, no worries. My plan is to check the demo vanilla project first and then try it in our project.

From: Dejan Jovasevic @.> Sent: Monday, October 9, 2023 2:10 PM To: DolbyIO/comms-sdk-unreal @.> Cc: Anton Iovbak @.>; Author @.> Subject: Re: [DolbyIO/comms-sdk-unreal] UE5 Server MAC build crash with DolbyIOModule error (Issue #415)

Note that we were also able to package/run using the 1.2.1 version of the plugin on macOS, so I am not sure that 1.2.2 changes will affect you. Likely you will be running into the same issue you had before. So to re-iterate something that we may not be understanding; what does it exactly mean UE5 Server MAC build? Is this a different way to package the game that we need to try? Or does it mean something else? Like from what I understand you are still using the plugin as intended for client application, so clarifying this part could be the part we are missing.

— Reply to this email directly, view it on GitHubhttps://github.com/DolbyIO/comms-sdk-unreal/issues/415#issuecomment-1752805449, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCQROF4KUPHJZWDAU4XRNWLX6PLR7AVCNFSM6AAAAAA47PXWKGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJSHAYDKNBUHE. You are receiving this because you authored the thread.Message ID: @.***>

djova-dolby commented 11 months ago

Yes my bad, these are all internal. I thought this second one was a public repository. But I see now it is all internal repositories that were never opened.

AntonIovbakDev commented 11 months ago

So, what is eventually? Could you just package sample and send to me via email or somehow to check? Or it is restricted on your side?

BR Anton

From: Dejan Jovasevic @.> Sent: Monday, October 9, 2023 2:12 PM To: DolbyIO/comms-sdk-unreal @.> Cc: Anton Iovbak @.>; Author @.> Subject: Re: [DolbyIO/comms-sdk-unreal] UE5 Server MAC build crash with DolbyIOModule error (Issue #415)

Yes my bad, these are all internal. I thought this second one was a public repository.

— Reply to this email directly, view it on GitHubhttps://github.com/DolbyIO/comms-sdk-unreal/issues/415#issuecomment-1752807854, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCQROF4EZMETTIFEQ2QXBUTX6PLYXAVCNFSM6AAAAAA47PXWKGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJSHAYDOOBVGQ. You are receiving this because you authored the thread.Message ID: @.***>

djova-dolby commented 11 months ago

No I cannot share the sample project. It has not been open sourced.

AntonIovbakDev commented 11 months ago

Hi guys!

As we've talked, I've downloaded dolby project https://github.com/DolbyIO/comms-sdk-unreal, created new branch for UpdateTexture fix, but I cannot push it. Seems like I don't have access to do that. Could you check pls?

[cid:6788cf88-4c9f-4c07-900f-b0053296264b]

Best regards, Engineering Lead Anton Iovbak

From: Dejan Jovasevic @.> Sent: Monday, October 9, 2023 2:51 PM To: DolbyIO/comms-sdk-unreal @.> Cc: Anton Iovbak @.>; Author @.> Subject: Re: [DolbyIO/comms-sdk-unreal] UE5 Server MAC build crash with DolbyIOModule error (Issue #415)

No I cannot share it. It has not been open sourced.

— Reply to this email directly, view it on GitHubhttps://github.com/DolbyIO/comms-sdk-unreal/issues/415#issuecomment-1752863046, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCQROF2OQZKJYK4J2UMY3V3X6PQMLAVCNFSM6AAAAAA47PXWKGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJSHA3DGMBUGY. You are receiving this because you authored the thread.Message ID: @.***>

djova-dolby commented 11 months ago

Yes you are correct it seems that the default permissions for the repo is to create PRs/Issues but can't push directly into repo. So in order to make the PR will in fact need to do the fork/PR workflow.

AntonIovbakDev commented 11 months ago

Hi!

Made PR

BR Anton Iovbak

From: Dejan Jovasevic @.> Sent: Monday, October 16, 2023 10:06 AM To: DolbyIO/comms-sdk-unreal @.> Cc: Anton Iovbak @.>; Author @.> Subject: Re: [DolbyIO/comms-sdk-unreal] UE5 Server MAC build crash with DolbyIOModule error (Issue #415)

Yes you are correct it seems that the default permissions for the repo is to create PRs/Issues but can't push directly into repo. So in order to make the PR will in fact need to do the fork/PR workflowhttps://gist.github.com/Chaser324/ce0505fbed06b947d962.

— Reply to this email directly, view it on GitHubhttps://github.com/DolbyIO/comms-sdk-unreal/issues/415#issuecomment-1763857928, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCQROF7WHROQDKCQFBZFZP3X7TMITANCNFSM6AAAAAA47PXWKE. You are receiving this because you authored the thread.Message ID: @.***>

AntonIovbakDev commented 10 months ago

Hi guys!

Could you give us an access to video processing handler config into plugin? As I see we have it in SDK. We want to restrict framerate for decrease net and CPU usage.

As I see you are using VideoHandler, but if want to change FPS we have to use VideoProcessingHandler and change config data. Am I right? How to set less FPS for all users? Maybe it is already there but I cannot find a way without plugin changes.

[cid:d5cd8444-a109-429b-82a3-9ec8330c4caa]

From: Dejan Jovasevic @.> Sent: Thursday, October 5, 2023 4:35 PM To: DolbyIO/comms-sdk-unreal @.> Cc: Anton Iovbak @.>; Author @.> Subject: Re: [DolbyIO/comms-sdk-unreal] UE5 Server MAC build crash with DolbyIOModule error (Issue #415)

Can you try with the artifacts from this run? https://github.com/DolbyIO/comms-sdk-unreal/actions/runs/6419413731 Also what is UE5 Server on MAC is this like a different way of packaging of Unreal or something like this? Google has not been helpful here. @Krysedhttps://github.com/Krysed was able to package game on macOS from what I understand with UE5.

— Reply to this email directly, view it on GitHubhttps://github.com/DolbyIO/comms-sdk-unreal/issues/415#issuecomment-1748914198, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCQROF3J3QELTUMJIRHD5NTX52ZRTAVCNFSM6AAAAAA47PXWKGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBYHEYTIMJZHA. You are receiving this because you authored the thread.Message ID: @.***>

AntonIovbakDev commented 10 months ago

From: Anton Iovbak @.> Sent: Thursday, November 9, 2023 12:05 PM To: DolbyIO/comms-sdk-unreal @.>; DolbyIO/comms-sdk-unreal @.> Cc: Author @.> Subject: Re: [DolbyIO/comms-sdk-unreal] UE5 Server MAC build crash with DolbyIOModule error (Issue #415)

Hi guys!

Could you give us an access to video processing handler config into plugin? As I see we have it in SDK. We want to restrict framerate for decrease net and CPU usage.

As I see you are using VideoHandler, but if want to change FPS we have to use VideoProcessingHandler and change config data. Am I right? How to set less FPS for all users? Maybe it is already there but I cannot find a way without plugin changes.

[cid:d5cd8444-a109-429b-82a3-9ec8330c4caa]

From: Dejan Jovasevic @.> Sent: Thursday, October 5, 2023 4:35 PM To: DolbyIO/comms-sdk-unreal @.> Cc: Anton Iovbak @.>; Author @.> Subject: Re: [DolbyIO/comms-sdk-unreal] UE5 Server MAC build crash with DolbyIOModule error (Issue #415)

Can you try with the artifacts from this run? https://github.com/DolbyIO/comms-sdk-unreal/actions/runs/6419413731 Also what is UE5 Server on MAC is this like a different way of packaging of Unreal or something like this? Google has not been helpful here. @Krysedhttps://github.com/Krysed was able to package game on macOS from what I understand with UE5.

— Reply to this email directly, view it on GitHubhttps://github.com/DolbyIO/comms-sdk-unreal/issues/415#issuecomment-1748914198, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCQROF3J3QELTUMJIRHD5NTX52ZRTAVCNFSM6AAAAAA47PXWKGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBYHEYTIMJZHA. You are receiving this because you authored the thread.Message ID: @.***>

AntonIovbakDev commented 10 months ago

Hi Jon,

If you are able to help on the below matter reported by Anton, that would be much appreciated as we need to restrict framerate with camera sharing as otherwise we have some quite severe performance issues.

All the best,

Patrick.

Patrick Wheeler CTO

[image001.png] +49 172 656 7345 [image002.png] cosociety.cohttps://cosociety.co/ [image003.png] Hamburg, Germany

[image004.gif]https://cosociety.co/

From: Anton Iovbak @.> Sent: Friday, November 10, 2023 9:22 AM To: Dejan Jovasevic @.> Cc: Patrick Wheeler @.>; Andrei Sabin @.>; DolbyIO/comms-sdk-unreal @.>; DolbyIO/comms-sdk-unreal @.> Subject: Fw: [DolbyIO/comms-sdk-unreal] UE5 Server MAC build crash with DolbyIOModule error (Issue #415)

From: Anton Iovbak @.**@.>> Sent: Thursday, November 9, 2023 12:05 PM To: DolbyIO/comms-sdk-unreal @.**@.>>; DolbyIO/comms-sdk-unreal @.**@.>> Cc: Author @.**@.>> Subject: Re: [DolbyIO/comms-sdk-unreal] UE5 Server MAC build crash with DolbyIOModule error (Issue #415)

Hi guys!

Could you give us an access to video processing handler config into plugin? As I see we have it in SDK. We want to restrict framerate for decrease net and CPU usage. As I see you are using VideoHandler, but if want to change FPS we have to use VideoProcessingHandler and change config data. Am I right? How to set less FPS for all users? Maybe it is already there but I cannot find a way without plugin changes.

@.***

From: Dejan Jovasevic @.**@.>> Sent: Thursday, October 5, 2023 4:35 PM To: DolbyIO/comms-sdk-unreal @.**@.>> Cc: Anton Iovbak @.**@.>>; Author @.**@.>> Subject: Re: [DolbyIO/comms-sdk-unreal] UE5 Server MAC build crash with DolbyIOModule error (Issue #415)

Can you try with the artifacts from this run? https://github.com/DolbyIO/comms-sdk-unreal/actions/runs/6419413731 Also what is UE5 Server on MAC is this like a different way of packaging of Unreal or something like this? Google has not been helpful here. @Krysedhttps://github.com/Krysed was able to package game on macOS from what I understand with UE5.

— Reply to this email directly, view it on GitHubhttps://github.com/DolbyIO/comms-sdk-unreal/issues/415#issuecomment-1748914198, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCQROF3J3QELTUMJIRHD5NTX52ZRTAVCNFSM6AAAAAA47PXWKGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBYHEYTIMJZHA. You are receiving this because you authored the thread.Message ID: @.**@.>>

djova-dolby commented 10 months ago

Hi, so we do not have a frame-rate control mechanism in the underlying c++sdk and unfortunately we will not be adding it as a new feature to the sdk in the near future. In general webrtc decides based on BWE the available bitrate what it can encode and at what frequency. The video processor plugin does not control the frame rate, it is actually using a Dolby Video Processing library to segment and blur the background of the capture image. There is no mechanism exposed from the c++sdk to just set desired frames, and thus nothing in the Unreal SDK. However you can use similar concept that video processor is using and you can implement a frame rate control mechanism rather simply yourself. The C++SDK provides the following start_video method which is accepting a device and video_frame_handler. This video frame handler can be implemented by any application, so it can receive frames from the sdk, process and inject them back, or just render them. Using this frame handler is essentially how Unreal SDK screen share local preview is done. So to implement frame rate control you can create a custom video frame handler (it must be done in the layer of Unreal SDK which interacts with c++sdk, something along the lines of the video processing handler). This video frame handler you create and provide to the start_video call will receive each frame from the SDK captured by the camera, this is done at 30 fps. Then you can just drop whatever frames you do not want to achieve your desired frame rate, so if you want 5fps, you will only provide back to the SDK every 6th frame. A basic c++ class example to achieve this would look something like:

#include <dolbyio/comms/media_engine/media_engine.h>
#include <memory>
#include <mutex>

class dummy_frame_dropper final
    : public dolbyio::comms::video_source,
      public dolbyio::comms::video_sink,
      public dolbyio::comms::video_frame_handler,
      public std::enable_shared_from_this<dummy_frame_dropper> {
 public:
  static constexpr int frame_rate_ = 30;
  explicit dummy_frame_dropper(int fps) : desired_frame_rate_(fps) {
    assert(desired_frame_rate_);
    frame_to_pass_back_ = frame_rate_/desired_frame_rate_;
  }
  ~dummy_frame_dropper() = default;
  void handle_frame(const dolbyio::comms::video_frame& frame) override {
    std::lock_guard<std::mutex> lock(sink_lock_);
    if (sdk_video_sink_) {
      // Only provide frame back to SDK if it the desired fps frame
      if (++frame_counter_ >= frame_to_pass_back_) {
        sdk_video_sink_->handle_frame(frame);
        frame_counter_ = 0;
      }
    }    
  }
  void set_sink(const std::shared_ptr<video_sink>& sink,
                const config&) override {
    std::lock_guard<std::mutex> lock(sink_lock_);
    sdk_video_sink_ = sink;
  }
  std::shared_ptr<video_sink> sink() override { return shared_from_this(); }
  std::shared_ptr<video_source> source() override { return shared_from_this(); }
 private:
  std::mutex sink_lock_;
  std::shared_ptr<dolbyio::comms::video_sink> sdk_video_sink_{};
  int desired_frame_rate_;
  int frame_to_pass_back_;
  int frame_counter_{0};
};

Or if you were to just make a copy of/edit the DolbyIOVideoProcessingFrameHandler and modify it a bit. You will need to pass it the desired frame rate that you would like. Then the handle frame function would look something like:

void handle_frame(const dolbyio::comms::video_frame& VideoFrame) override {
  if (SdkSink) {
    // Only pass frams to the SDK sink for desired frame rate
    if (++frame_counter_ >= frame_to_pass_back_) {
      SdkSink->handle_frame(VideoFrame);
      frame_counter_ = 0;
    }
    // Preview the frames at 30fps
    PreviewSink->handle_frame(VideoFrame);
  }
}