[Video Player] Support gstreamer dmabuf in video_player

[makotosato-at]

Hello. I have i.MX8MP board and using video_player. This SoC has a VPU(Video Proccessing Unit) so that it can hardware decode H.264 with gstreamer. But, 1080p 30fps or 60fps video, the following _gst_bufferextract() is the bottleneck and playback is slow. (_gst_extractbuffer() takes about 60ms.) gst_video_player.cc:

const uint8_t* GstVideoPlayer::GetFrameBuffer() {
  std::shared_lock<std::shared_mutex> lock(mutex_buffer_);
  if (!gst_.buffer) {
    return nullptr;
  }

  const uint32_t pixel_bytes = width_ * height_ * 4;
  gst_buffer_extract(gst_.buffer, 0, pixels_.get(), pixel_bytes);
  return reinterpret_cast<const uint8_t*>(pixels_.get());
}

I think the availability of dmabuf would solve this problem. Are there any plans to support gstreamer dmabuf in video_player? (In my environment, _gst_is_dmabufmemory() returns TRUE, so I can use dmabuf.)

[HidenoriMatsubayashi]

We have two options here to improve the performance. The first option is dmabuf you mentioned above. The other is GPU texture (see also https://github.com/sony/flutter-embedded-linux/issues/157).

I think the availability of dmabuf would solve this problem. Are there any plans to support gstreamer dmabuf in video_player?

I think that's good. If you have a patch for that, can you please send a PR?

[Kofhein]

@makotosato-at , you may try to set qos to TRUE, for me it improved playback and also resolved issue with disappearing sound. But for sure 60ms bottleneck is problem that should be solved.

[makotosato-at]

@Kofhein Thank you for the information. I tried that, but it didn't solve the problem...

@HidenoriMatsubayashi Sorry, I don't have any patches...

[Kofhein]

@makotosato-at , my comment might be helpful to you too https://github.com/sony/flutter-elinux-plugins/issues/63#issuecomment-1399358789 I've resolved similar issue, but in my case I've got desktop HW, so in my case gst_buffer_extract wasn't bottleneck. Also I've seen dma buf based implementation here https://github.com/ardera/flutter-pi/blob/master/src/plugins/gstreamer_video_player/player.c , but it's C based solution for RasbPi and seems there GPU textures already implemented. In most cases I've seen this kind of dmabuf implementation it was connected to GPU texture using Frame buffer and rendering buffer with planes.