Closed sotelo closed 3 months ago
Hi @sotelo Thank you for the kind talk.
1) Please take a look at tests/test_PySurface.py unit test file. Within it, conversion between Surface
and Tensor
is tested in both directions.
2) Colour conversion is done with the help of NPP library which unfortunately doesn’t support (at least at the moment) conversion from rgb to nv12. However, you can convert rgb to yuv444 and give that surface to encoder. It does support yuv444 just fine. The only thing you need to do is to specify yuv444 pixel format in PyNvEncoder
constructor.
Thanks for the quick answer @RomanArzumanyan!
I was able to successfully convert between Surface
and Tensor
after looking at the code example. However, when I try to convert the surface from rgb to yuv444 I get the error: TaskExecInfo.UNSUPPORTED_FMT_CONV_PARAMS
.
Here's my code (using the video in the test examples):
import PyNvCodec as nvc
# url = "/data/jose/Turnbull.mp4"
url = "/u/home/code/VALI/tests/data/test.mp4"
class StopExecution(Exception):
def _render_traceback_(self):
return []
# GPU-accelerated decoder
pyDec = nvc.PyDecoder(
url,
{},
gpu_id=0)
# GPU-accelerated converter
pyCvt = nvc.PySurfaceConverter(
pyDec.Format(),
nvc.PixelFormat.RGB,
gpu_id=0)
# Colorspace conversion context. Here a dummy set of parameters is used
# just for illustration purposes.
cc_ctx = nvc.ColorspaceConversionContext(
nvc.ColorSpace.BT_709,
nvc.ColorRange.MPEG)
# Raw decoded Surface
surf_src = nvc.Surface.Make(
format=pyDec.Format(),
width=pyDec.Width(),
height=pyDec.Height(),
gpu_id=0)
# Raw Surface, converted to RGB
surf_dst = nvc.Surface.Make(
format=nvc.PixelFormat.RGB,
width=pyDec.Width(),
height=pyDec.Height(),
gpu_id=0)
surface_yuv444 = nvc.Surface.Make(
format=nvc.PixelFormat.YUV444,
width=pyDec.Width(),
height=pyDec.Height(),
gpu_id=0,
)
to_yuv444 = nvc.PySurfaceConverter(
nvc.PixelFormat.RGB,
nvc.PixelFormat.YUV444,
gpu_id=0,
)
success, details = pyDec.DecodeSingleSurface(surf_src)
if not success:
print(details)
# Convert tot RGB
success, details = pyCvt.Run(surf_src, surf_dst, cc_ctx)
if not success:
print(details)
frame = torch.from_dlpack(surf_dst)
frame = torch.reshape(frame, (pyDec.Height(), pyDec.Width() * 3))
surface = nvc.Surface.from_dlpack(
torch.utils.dlpack.to_dlpack(frame), nvc.PixelFormat.RGB)
success, details = to_yuv444.Run(surface, surface_yuv444, cc_ctx)
print(success)
print(details)
Hi @sotelo
That’s the usual color conversion parameters business. Only JPEG cooor range is supported in rgb 2 yuv444 conversion by npp: https://github.com/RomanArzumanyan/VALI/blob/c4ce78d5e06beda2ada8f680078044b8cd3d4654/src/TC/src/TaskConvertSurface.cpp#L654
I should probably make cconv params argument completely optional and use default supported parameters if nothing is given by user.
Cconv parameters support was intended to improve color conversion accuracy but it looks to me now that it should be optional and not mandatory.
@sotelo
Please LMK if the issue is resolved and can be closed.
Yes, thank you.
Hello @RomanArzumanyan,
First of all, thank you for continuing your work in VALI! I think that it's an amazingly useful piece of software. In my application, I used to use VPF for reading, processing (with PyTorch) and writing a video without moving the data to CPU. I'm porting my code to VALI and there's a couple of questions that I have in the process:
torch
into anvc.Surface
object for encoding?nvc.PixelFormat.RGB
intonvc.PixelFormat.NV12
to write back the video?We get
ValueError: Unsupported pixel format conversion
with this code: