Texture/Material displays a streched Image based on Aspect Ratio

[MaxDorn96]

Hello dear Dolby/Millicast Team.

We are currently facing the Issue that, if we Share the Screen of a Person which isn't 16:9 the Texture gets streched on the Plane which is scaled to be 16:9.

I wanted to ask if there is a way to recieve the Height/Width or Aspect Ratio directly from the MillicastTexture2DPlayer or the MillicastTexture2D since I couldnt find a function for that at the moment.

We either want the Texture to not be stretched aka keeps it's Aspect Ratio on the Plane we use the Texture on or we could, if we are able to get the Resolution or Aspect Ratio somehow, scale the Plane accordingly I suppose.

I dont know if anything like that is already possible but I wasnt able to find anything like that in the Documentation site directly.

Thank you for all your hard work!

[dbaldassi]

Hi, You should be able to get the width and height of the stream by looking at the player stats. Does it happen only with a screen capture ? or any video feed ?

[MaxDorn96]

Sorry but I can't seem to find the Player Stats you are Talking about, in which Component would I find such a Function or Variable, the Documentation also doesn't seem to mention it directly?

I only tried getting the Width/Height from the VideoTexture oder Texture2DPLayer, but they dont seem to have such functions.

Am I just missing something here?

But if I can access the data as you said above I can certainly calculate the Ratio and use this then.

I believe it happened in any Videofeed or rather when we shared our Screen or Tab since on our end we have an 21:9 screen and cupturing that but displaying that in a Texture on a Plane which is 16:9 so we just need to dynamically, everytime when someone streams be able to get the Height/Width as you mentioned to calculate and change the ratio/size of the plane I suppose.

So where would I find these Player Stats exactly? (Sorry that im not that advanced in this yet)

[rweber89]

Hi @MaxDorn96 the PlayerStats have recently been exposed in the SubscriberComponent. On the dev branch that is.

[MaxDorn96]

Thanks for the Info, just saw in the Repo that im behind on that. Will get the newest version of the Plugin and check everything thats changed and hopefully see the Stats just as you mentioned.

[MaxDorn96]

I am kind of confused how I am suppose to work with the new AudioComponent, so basically what the Intended way is. Since it is an SceneComponent it doesn't seem like I can just add it to my Actor Blueprint via Add Component, so I would have to do so on runtime via AddComponent (If Im not mistaken)

And If I would add it like in the First picture on runtime via AddComponent I can't see what to do now. I know I can add this as an Consumer by OnAudio but where would I be able to Setup the SoundClass and Attenuation like before within my Blueprint? Since the Docs aren't mentioning this new System yet I am kinda at a loss of what to do now in order to be able to have single Instances of AudioComponents for each BP_MillicastScreen I place in my Level in which I can individually change Spatalization or SoundClass etc.

Furthermore in Blueprint I can't seem to be able to get any PlayerStats from the Subscriber Component. Could it be that they aren't Blueprint exposed?

[rweber89]

Hi @MaxDorn96 those are multiple issues, so I`ll address them separately.

Exposure of PlayerStats to Blueprint is not really an option right now. Would you mind letting me know what the information you are looking for and how critical this is to you?

Regarding the AudioComponents. That has nothing to do with it being a SceneComponent, but rather there was an oversight in the Meta tags with it not being BlueprintSpawnable. However I have overhauled the system. As we are working towards making the plugin more flexible and less imposing on architectural choices. ( https://github.com/millicast/millicast-player-unreal-engine-plugin/commit/0455771ec1801838461b8f2c9069d11b5c1c5058 )

If you have a MillicastActor in your level you can assign it MillicastAudioActors or AudioComponents to its AudioConsumer variables. There is one array for each type. ( on the example project dev branch. https://github.com/millicast/millicast-player-ue5-example/commit/e52a75fa93fe6e3838dc0b8c507d613aedc57ef7). It will then handle the necessary steps for you. However if you wanted to do it on the fly yourself, the C++ code shows you how. You would listen to the Subscribers OnAudioTrack event, and from there access the MillicastAudioSubsystem (you can do this via Blueprint!) and first call Register with your AudioComponent on the Subsystem. Then call AddConsumer on the AudioTrack with the result of the subsystems GetInstance(AudioComponent) function.

[MaxDorn96]

Hello.

I would like to be able to call a Function from somewhere in which I would get the Resolution or Width/Height/Apsect Ratio of a Stream. I would most likely call this at every OnActive since this is the Event used when a new Stream is detected.

If someone with an 21:9 Monitor streams onto our Plane in the Level, which is scaled to be 16:9, the Image will be squished which makes sense since an 21:9 texture/stream is tried to be displayed on an 16:9 surface.

So I would like to be able to get the Ratio/Resolution/Width/Height from an Function easily at any point in time to either scale the Plane or maybe do adjustments on the Material/Texture I suppose to be able to allow any Aspect Ratio.

It could be that your Update on the AudioActor maybe solved my Problem. What I want basically is anything, like an Component or ChildActor, which I can just Add to my Blueprint and Adjust some default Values like SoundClass and Attenuation, like it did up to this point with an ChildActor which holds the MillicastAduioActor Class.

So If I can just assign/add the MillicastAudioCOmponent to my Actor Blueprint and beeing able to Change the Attenuation and SoundClass there then all my needs are fullfilled basically.

[rweber89]

Hi @MaxDorn96 please let me know about the audio side. I´ll look at exposing those specific stats for you.

[MaxDorn96]

Im still kind of lost since I still can't add any sort of MillicastAudioComponent to my Blueprint (I can't even Add it vie AddComponentByClass anymore) and I only have an MillicastAudioActor as before in my Blueprint which I can't add as an Consumer.

What exactly are the steps I would need to make the system work If I were to create a new Blueprint which should hold an AudioConsumer to add to the OnAudio Event?

Like I said up to this point I just had a Child Actor in my Blueprint of Class MillicastAuioActor in which I set all default Values for my BP (Like SoundClass and Attenuation)

[rweber89]

Hi @MaxDorn96

Thank you for your valuable questions and feedback. I have simplified the system some more and increased usability.

For it to work do the following:

Add a UAudioComponent to any actor where you would like to have your audio source for the stream.

Register the AudioComponent before you call Subscribe on your Millicast Subscriber.

The AudioComponent does not need to be in the same actor as your SubscriberComponent (!) though it usually is.

Please let me know if that works for you.

[MaxDorn96]

Oh that looks great, I will try that ASAP

[MaxDorn96]

Okay, I just were able to test it and it worked perfectly.

I guess the OnAudio Event is rather useless for us now since we dont have anything to add as an consumer I assume? Now Im only missing the Width/Height Function in some way and all requirements for our next Update are set xD

[rweber89]

@MaxDorn96 you can access the data now by calling GetStats on the Subscriber Component

[MaxDorn96]

Hey, thanks for that!

I added this here on the OnActive Event (Aka when a stream is detected) and it seems based on the Editor all values from the GetStats are always 0.

I build in a loop to check if it just needs a second but the values always stay 0 so far

I used the Millicast or rather Dolby Boradcast Feature with the simple "Start Video Test" to check if the intended Data comes through.

I can see the stream as usual but just the GetStats doesnt return anything yet

[rweber89]

Hi @MaxDorn96 - what UE version are you working with?

[MaxDorn96]

I am using the Source Engine and im on the 5.1 branch so I assume 5.1.0 or 5.1.1

[rweber89]

@MaxDorn96 I am sorry I will not be able to get back to you about this probably before next week. Is this happening on an actual stream? Do you have any access to C++ with your project? Which would help with debugging the issue.

[MaxDorn96]

Sorry I am not good debugging with C++ sadly.

I was able to test this inside the Ediotr. I basically started a stream via Broadcast on the DolbyIO site which triggered the OnActive Event correctly and the stream got displayed just as intended.

After OnActive has been called, which I was able to ensure via Breakpoints in Blueprints i kept looping the GetStats call (Becase maybe it needs a second or two to correctly display the Data). I was printing the Stats from GetStats in the Loop (The loop i built basically looked if width is 0 and if yes, try once more after a Delay) and all stats were always Empty or 0 based on my Print Strings and Breakpoints, so even tho the Stream displayed correctly and nothing was wrong with it just the stats seem to have been empty.

I've tested it with the Start Video Test in the Dolby Broadcast Function which sends a stream with an Video/Frame Test as always

[rweber89]

I will take a look at this next week.

[MaxDorn96]

Do you have updates on this regard yet or does it still seem that it should work as it is?

[rweber89]

Hi @MaxDorn96 I am sorry I was working on another issue that took considerably longer than expected

[rweber89]

@MaxDorn96 On the dev branch there are two options now. You have an event (OnVideoResolutionChanged) that you can bind to in Blueprint either on the Track (Which you can get from OnVideoTrack) or on the VideoConsumer which you add (as long as it inherits from UMillicastTexture2DPlayer)

[MaxDorn96]

Will try this soon, sounds awesome, can't wait to try it xD

[MaxDorn96]

I have to say it works like a charm, but there is just like a slight Issue.

The Evenet Resolution Changed wont be called when I first recieve the stream, which makes sense I guess since I didnt change the resolution yet. It doesnt matter if I bind the Event on the OnVideo or OnActive track, but once one a change is made to the size it will send it perfectly.

Based on the breakpoints he always goes to bind that event but it wont get called until I actually change the resolution.

So I basically need the Width+Height for the first time when recieving the stream basically, so whenever OnActive is called I can bind on ResolutionChanged but also use the Width+Height once to set the Size once.

So basically I would need a Function where I manually get the WIdth+Height once I guess so on every OnActive I could set it once and the Bind will handle the rest, I dont know if this would be easily implementable by exposing the width/height variables as UPROPERTIES or making a seperate Function, but as of now this new Event covers everything besides the Init so to speak

[rweber89]

@MaxDorn96 sorry this was a while ago and I may have forgotten to comment on the issue. I did address this, right?

[MaxDorn96]

As far as Im concerned you addressed this about 99%.

The Event I can bind on called OnVideoResolutionChanged works, but only if the person streaming changes it while im already subscribed.

So If I join in a level in which somebody already streams and I subscribe the OnVideoResolutionChanged doesnt seem to get called und the Video stays streched for me till he adjust his resolution.

So I kind of need a way in which OnVideoResolutionChanged will get called once manually whenever OnActive is getting called aka I recieve the stream or even better for me would be an Event/Function I can call Manually to return the Width and Height Values which are returned by the OnVideoResolutionChanged Delegate.

I hope this makes sense to you and won't be too hard to pull off, this would be the last thing that kind of is missing at the moment on my end

[MaxDorn96]

I really dont want to bother anyone, I just wanted to ask what the Status of this Issue/Request of mine currently is, because since the new assignment I didn't see any answer or updates to any of my Issues and I can't see Updates regarding this, or even a bit in general, on Github.

[dbaldassi]

Hi, sorry for the delay, I'll take a look

[dbaldassi]

Do you need something like this ? If, yes I'll just push what I have in local

[MaxDorn96]

Sorry for the late answer.

It seems like the Functions outputs a Vector2 right, so I would have the Width and Height, just like the Event OnVideoResolutionChanged I can bind on, right?

If it is than thats exactly what I need, I basically only need a Function I can call which gives me the current Width and Height of the Stream, looks like thats exactly what you have build.

[dbaldassi]

Great, you can try it in the dev branch then 👍

[dbaldassi]

Hi, did you have some time to test it ? If yes, is it ok to close this ?