TimmHess / UnrealImageCapture

A small tutorial repository on capturing images with semantic annotation from UnrealEngine to disk.
MIT License
225 stars 51 forks source link

Passing and saving a UTextureRenderTarget2D object. #29

Open jmatthess opened 1 year ago

jmatthess commented 1 year ago

Hey i thought it would be better to make a new issue for people to better find this than my question being embedded into an unrelated issue topic.

I wanted to ask though @TimmHess. Would it require a lot of rewriting and editing to make it save a UTextureRenderTarget2D object instead of saving what is currently shown via the capture2d camera? Long story short i'm wanting to send my Composure renders over the network but my first goal is to at least get it to save to a jpeg without issue first. Composure has a lovely virtual override function called "RelayOutput_Implementation" where i can get the UTextureRenderTarget2D object. I tried to simply slot in the object into your code hoping it would work but i'm still newish to C++ and UE and have been learning it through trial by fire at this point but when it comes to sending requests to the GPU, that's when i get a bit out of my depth.

added input for a UTextureRenderTarget2D and tried to use that for the renderTargetResource variable image

But on further inspection i think i was going about this all wrong as the UTextureRenderTarget2D already has an image and i'm not needing to request an image from the GPU as that's all already handled by Composure. Below is an error i get with my current code, as well as the line where it breaks. But again i assume this is because i'm not requesting the image from the gpu like your current setup does. image image

I can see that when you use GetData on line 118 you have nextRenderRequest->Image.GetData() which returns a TArray which i assume is the data type that's needed for this to work. Can you see a way for this to work if i already have a UTextureRenderTarget2D object? The ReadPixels function is so ungodly slow and assume the Image.GetData() is a much faster process in comparison.

TimmHess commented 1 year ago

Hey, this time I was on vacation :see_no_evil:

Indeed, ReadPixels() is so slow because it runs a pipeline flush on the GPU, I guess to determine the point in time where the buffer can actually be read. (At least it did so when I first wrote this code and I did not check whether it still does) So the whole RenderFence stuff is just to replace the flush but if I do remember correctly most of the code is actually a copy of ReadPixels.

Hence, I would have believed that what you did should work. What I do not see from your screenshots is the format initialization of the CompsureTextureTarget. I would hope that the crash is not because the buffer (texture) is empty but because the wrapper expects a different format. Can you validate that the buffer is (most importantly) not null and actually contains anything meaningful? At least has the right size? Then you can either try to figure out what the data format is for the TextureTarget and adjust the wrapper, or overwrite the TextureTarget's settings hoping that it was not too important at another position in the unreal codebase.