When I used RawVideo dataloader, I faced one problem inside decord

[cecilia930426]

[h264 @ 0x562e0e0b0d00] mmco: unref short failure For almost all the videos, I faced this problem. And I don't know how to solve it.

[innerlee]

Which version of decord is used? And please attach a sample video for debugging

[cecilia930426]

Which version of decord is used? And please attach a sample video for debugging

The version is 0.4.0. The video id is v_V3dTp7_NyiE on Activitynet dataset. Almost all the video on activitynet dataset have this kind of problem.

[innerlee]

Please try opencv decode, and see if it can work

[cecilia930426]

Please try opencv decode, and see if it can work

How to use that when training?

[innerlee]

change the config file by replacing decord with opencv

[irvingzhang0512]

modify your config file.

DecordInit -> OpenCVInit DecordDecode -> OpenCVDecode

[innerlee]

there is also a pyav backend

[cecilia930426]

modify your config file.

DecordInit -> OpenCVInit DecordDecode -> OpenCVDecode

I used OpenCV, the same issues happened

[innerlee]

So it is a problem of the video. How were they generated?

[irvingzhang0512]

@innerlee There are many issues related to video deocdors and broken videos. Maybe we should add this to faq

[cecilia930426]

So it is a problem of the video. How were they generated?

It is not the problem with video, because it downloaded from youtube. Also, I run the decord official code and don't get the same errors. decord

There is maybe sth wrong in your decode video process, or sth wrong with my setting.

from decord import VideoReader
from decord import cpu, gpu

vr = VideoReader('examples/flipping_a_pancake.mkv', ctx=cpu(0))
# a file like object works as well, for in-memory decoding
with open('examples/flipping_a_pancake.mkv', 'rb') as f:
  vr = VideoReader(f, ctx=cpu(0))
print('video frames:', len(vr))
# 1. the simplest way is to directly access frames
for i in range(len(vr)):
    # the video reader will handle seeking and skipping in the most efficient manner
    frame = vr[i]
    print(frame.shape)

# To get multiple frames at once, use get_batch
# this is the efficient way to obtain a long list of frames
frames = vr.get_batch([1, 3, 5, 7, 9])
print(frames.shape)
# (5, 240, 320, 3)
# duplicate frame indices will be accepted and handled internally to avoid duplicate decoding
frames2 = vr.get_batch([1, 2, 3, 2, 3, 4, 3, 4, 5]).asnumpy()
print(frames2.shape)
# (9, 240, 320, 3)

# 2. you can do cv2 style reading as well
# skip 100 frames
vr.skip_frames(100)
# seek to start
vr.seek(0)
batch = vr.next()
print('frame shape:', batch.shape)
print('numpy frames:', batch.asnumpy())

[innerlee]

What's the fps the the problematic videos? Check if there are problems with 30fps

[cecilia930426]

What's the fps the the problematic videos? Check if there are problems with 30fps

Almost all of them have errors, including 30fps.

[dreamerlin]

It seems Decord runs successfully in most cases, you may need to check your videos, decord version or your environments

[owaisCS]

The same issue is propping up for almost all videos : [h264 @ 0x13ba10370] mmco: unref short failure For ActivityNet. I am using Decord 0.6.0 Following is the train pipeline I am using: train_pipeline = [ dict(type='DecordInit'), dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8), dict(type='DecordDecode'), dict(type='Resize', scale=(-1, 256)), dict( type='MultiScaleCrop', input_size=224, scales=(1, 0.875, 0.75, 0.66), random_crop=False, max_wh_scale_gap=1, num_fixed_crops=13), dict(type='Resize', scale=(224, 224), keep_ratio=False), dict(type='Flip', flip_ratio=0.5), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False), dict(type='FormatShape', input_format='NCHW'), dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]), dict(type='ToTensor', keys=['imgs', 'label']) ]