whether one decoder output to one display pipeline and one encoder pipeline is possible with sample_multi_transcode or any other samples in msdk

[APrashant]

Hi,

To do: Aim is to achieve 1 - N pipeline like input - one decoder to 1 - vpp - display pipeline 2 - vpp - encoder - filedump pipeline then same above module for 16 channel

Tried below par files:

1. below is 1-N - encoder pipeline and it is working

-i::h264 h264.h264 -o::sink -hw -i::source -w 320 -h 240 -o::h264 output1.h264 -async 1 -hw -i::source -o::h264 output4.h264 -async 1 -hw -i::source -o::h264 output3.h264 -async 1 -hw -i::source -o::h264 output4.h264 -async 1

added above lines in a par file - filename.par and executed ./sample_multi_transcode -par filename.par

2. Also tried display using vpp_comp_only option

-i::h264 test.h264 -dec_postproc -w 1920 -h 1080 -join -hw -async 1 -o::sink -vpp_comp_dst_x 0 -vpp_comp_dst_y 0 -vpp_comp_dst_w 1920 -vpp_comp_dst_h 1080 -ext_allocator -vpp_comp_only 1 -w 1920 -h 1080 -async 1 -join -hw -i::source -ext_allocator -ec::rgb4 -rdrm-DisplayPort -fps 30

the above par file plays decoded video in display in the given x y position

Question : can we achieve something like below pipeline:

sample_multi_transcode par file: -i::h264 h264.h264 -o::sink -hw -i::source -(encoder pipeline) -hw -i::source -(display pipeline)

Can you please suggest any sample already we have to achieve this or with sample_multi_transcode any par file we have !

Thanks in advance.

[APrashant]

Hi @SofyaBalandina ,

with the above display pipeline we used vpp_comp option along with vpp_comp_only option to have filedump.

par file used: -i::h264 test.h264 -join -hw -async 1 -o::sink -vpp_comp_dst_x 0 -vpp_comp_dst_y 0 -vpp_comp_dst_w 320 -vpp_comp_dst_h 240 -ext_allocator -vpp_comp 1 -async 1 -join -hw -i::source -o::raw vpp_dump_file.yuv -vpp_comp 1 -join -hw -i::source -o::h264 encode_dump_file.h264 -async 1 -vpp_comp_only 1 -async 1 -join -hw -i::source -ext_allocator -ec::rgb4 -rx11 -fps 10

Observation: par file passed and transcoding started display also happening yuv file dump is working encoded dump is working

Issue: Display is not in the normal speed - it is in 4x speed and display window stuck after last frame and getting below errors after some time.

Error log: mamatha@Rental32-LaptopU:~/msdk/build/__bin/release$ ./sample_multi_transcode -par parfile.par Multi Transcoding Sample Version 8.4.27.0

Par file is: trail.par

External allocator will be used as some cmd line paremeters request it. Some inter-sessions do not use opaque memory (possibly because of -o::raw). Opaque memory in all inter-sessions is disabled. libva info: VA-API version 1.13.0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so libva info: Found init function __vaDriverInit_1_13 libva info: va_openDriver() returns 0 Session 0: Input video: AVC Output video: To child session

Session 1: Pipeline surfaces number (EncPool): 1 Input video: From parent session Output video: DUMP

Session 2: Pipeline surfaces number (EncPool): 4 Input video: From parent session Output video: AVC

Session 3: Pipeline surfaces number (EncPool): 2 Input video: From parent session Output video: To child session

Pipeline surfaces number (DecPool): 5 Session 0 was joined with other sessions Session 1 was joined with other sessions Session 2 was joined with other sessions Session 3 was joined with other sessions

Transcoding started

(Observation: Transcoding started and two file dumps are succesfull, But issue with display (Playing with high speed and video gets paused), after some time execution terminated with below errors.)

..............ERROR: timed out waiting surface from upstream component

[ERROR], sts=MFX_ERR_NOT_FOUND(-9), Run, CTranscodingPipeline::Run::Encode() [0x5563f581fbd0] failed at /home/mamatha/msdk/samples/sample_multi_transcode/src/pipeline_transcode.cpp:4621

session 1 [0x5563f581fbd0] failed with status MFX_ERR_NOT_FOUND shutting down the application...

session [0x5563f57dfe20] m_bForceStop is set session [0x5563f581fbd0] m_bForceStop is set session [0x5563f58ba5b0] m_bForceStop is set session [0x5563f594c9c0] m_bForceStop is set ERROR: timed out waiting surface from upstream component

[ERROR], sts=MFX_ERR_NOT_FOUND(-9), Run, CTranscodingPipeline::Run::Encode() [0x5563f594c9c0] failed at /home/mamatha/msdk/samples/sample_multi_transcode/src/pipeline_transcode.cpp:4621

session 3 [0x5563f594c9c0] failed with status MFX_ERR_NOT_FOUND shutting down the application...

session [0x5563f57dfe20] m_bForceStop is set session [0x5563f581fbd0] m_bForceStop is set session [0x5563f58ba5b0] m_bForceStop is set session [0x5563f594c9c0] m_bForceStop is set

Transcoding finished

Common transcoding time is 304.804 sec

*** session 0 [0x5563f57dfe20] PASSED (MFX_ERR_NONE) 4.75315 sec, 1398 frames, 294.121 fps -i::h264 test.h264 -join -hw -async 1 -o::sink -vpp_comp_dst_x 270 -vpp_comp_dst_y 270 -vpp_comp_dst_w 320 -vpp_comp_dst_h 240 -ext_allocator

*** session 1 [0x5563f581fbd0] FAILED (MFX_ERR_NOT_FOUND) 304.777 sec, 0 frames, 0.000 fps -vpp_comp 1 -async 1 -join -hw -i::source -o::raw vpp_dump_file.yuv

*** session 2 [0x5563f58ba5b0] PASSED (MFX_ERR_NONE) 4.77596 sec, 1245 frames, 260.680 fps -vpp_comp 1 -join -hw -i::source -o::h264 encode_dump_file.h264 -async 1

*** session 3 [0x5563f594c9c0] FAILED (MFX_ERR_NOT_FOUND) 304.804 sec, 0 frames, 0.000 fps -vpp_comp_only 1 -async 1 -join -hw -i::source -ext_allocator -ec::rgb4 -rx11 -fps 10

---

The test FAILED

[ERROR], sts=MFX_ERR_NOT_FOUND(-9), main, transcode.ProcessResult failed at /home/mamatha/msdk/samples/sample_multi_transcode/src/sample_multi_transcode.cpp:1255

Any change needed in the parfile to play video in proper speed and fps ?

Thanks in advance

[sbalandi]

Hi, Sorry, I can’t give advice about about it, the vpp_comp options are target for n->1 scenarios, it is not expected that it will work with 1->N and here it is looks like surfaces handling work incorrectly Now in sample_multi_transcode is not presented some way to combine rendering and dumping. For rendering it is could be used sample_decode with -r options. Look like rendering to smt could be added with RenderFrame from device, if on configuration step put to Init nViews as 1

[APrashant]

Hi @SofyaBalandina

Thanks for that reply. This helps us.

We are planning to approach two ways to achieve 1 - N pipeline like input - one decoder to i) above decoder output - vpp - display pipeline ii) above decoder output - vpp - encoder - filedump pipeline

one way: -i::h264 h264.h264 -o::sink -hw -i::source -drm/x11 (display) -hw -i::source -o::h264 output4.h264 -async 1

Is that possible to sync both display and encoding in 1-N pipeline by implementing in this way! And whether we have any code reference for this type

Another way: Debugging this below pipeline itself.

-i::h264 test.h264 -join -hw -async 1 -o::sink -vpp_comp_dst_x 0 -vpp_comp_dst_y 0 -vpp_comp_dst_w 320 -vpp_comp_dst_h 240 -ext_allocator -vpp_comp 1 -async 1 -join -hw -i::source -o::raw vpp_dump_file.yuv -vpp_comp 1 -join -hw -i::source -o::h264 encode_dump_file.h264 -async 1 -vpp_comp_only 1 -async 1 -join -hw -i::source -ext_allocator -ec::rgb4 -rx11 -fps 10

can you suggest which way of implementation will be fine to continue based on the surface memory and sync operation perspective.

Also is there any better approach to implement and achieve our pipeline

Thanks in advance

[sbalandi]

Hi, Is that possible to sync both display and encoding in 1-N pipeline by implementing in this way? - I suppose that, yes, but to answer for sure need to do experiments And whether we have any code reference for this type ? - no, now we have no such code Looks like code for vpp_comp/vpp_comp_only close to solving the problem https://github.com/Intel-Media-SDK/MediaSDK/blob/master/samples/sample_multi_transcode/src/pipeline_transcode.cpp#L1277, but something went wrong Could you, please, try without dumping to raw you, without session 1: -vpp_comp 1 -async 1 -join -hw -i::source -o::raw vpp_dump_file.yuv ?

[APrashant]

Hi @SofyaBalandina,

Thank you so much for the prompt reply.

we tried as you mentioned above, -i::h264 1min.h264 -hw -async 1 -vpp_comp_dst_x 0 -vpp_comp_dst_y 0 -vpp_comp_dst_w 1280 -vpp_comp_dst_h 720 -o::sink -ext_allocator -vpp_comp 1 -join -hw -i::source -o::h264 encode_dump_file.h264 -async 1 -vpp_comp_only 1 -w 1280 -h 720 -async 1 -hw -i::source -ec::rgb4 -rx11 -fps 30

But no improvement in display. same issue observed. (Display is not in the normal speed - it is in 4x speed and display window stuck after last frame)

After that we tried without h264 dump and with yuv dump: -i::h264 1min.h264 -hw -async 1 -vpp_comp_dst_x 0 -vpp_comp_dst_y 0 -vpp_comp_dst_w 1280 -vpp_comp_dst_h 720 -o::sink -ext_allocator -vpp_comp 1 -w 1280 -h 720 -async 1 -hw -i::source -o::raw vpp_dump_file.yuv -vpp_comp_only 1 -w 1280 -h 720 -async 1 -hw -i::source -ec::rgb4 -rx11 -fps 30

Observation: In this case, Display was playing better than previous observations. (1min video played in 46 sec) Note: When we decrease resolution in the above yuv dump pipeline the speed of the display was increased Decreasing resolution indirectly proportional to display speed.

We couldn't figure out why resolution of dump pipeline affects the display pipeline speed. Can you please help us on this.

Thanks in advance.

[sbalandi]

Does the sample_multi_transcode terminate with an error(The test FAILED) or with The test PASSED without session with output to yuv? I'll check what could be wrong with display.