JNoDuq / videobench

VMAF PSNR Bitrate Analyzer
226 stars 43 forks source link

[Bug] KeyError: 'duration' #39

Closed polarathene closed 1 year ago

polarathene commented 2 years ago

The progress reached 40% but failed after processing for a while with the following:

*****************************
STARTING VIDEO BENCH
python3 /tmp/videobench.py -ref /tmp/video/a.mp4 -i /tmp/video/b.mkv -sync 0 -sw 0.0 -deint auto -subsampling auto -scale neighbor -vmaf_model auto -loglevel quiet
*****************************
* Analyzing Reference File...

-> a.mp4 
 Bitrate  : 8.67 Mbps
 avg_framerate : 60.0
 interlaced : 0 

* Analyzing tests Files...

Traceback (most recent call last):
  File "/tmp/videobench.py", line 183, in <module>

    list_input_obj = manage_input_files(all_input, loglevel)
  File "/tmp/videobench.py", line 83, in manage_input_files
    input_obj.duration = float(ffprobe_json['streams'][0]['duration'])
KeyError: 'duration'
polarathene commented 2 years ago

I had a lengthy response but lost it to a system crash. Still I'll post changes required here, looking at PR activity and maintainer interaction lacking on open PRs, I'm hesitant to create a new PR.

This KeyError occured due to the media stream for b.mkv not having a duration property, but the format object sibling of streams object array does have the expected duration value. You could use that instead, otherwise perhaps it's better as a fallback?:

https://github.com/JNoDuq/videobench/blob/ea447031bfcc0315a4ae7d793896f051cc8bc2c3/videobench.py#L83

            try:
                duration = ffprobe_json['streams'][0]['duration']
            except KeyError:
                duration = ffprobe_json['format']['duration']

That won't work though as format object is not available unless we also query that here:

https://github.com/JNoDuq/videobench/blob/ea447031bfcc0315a4ae7d793896f051cc8bc2c3/videobench_functions.py#L266-L267

It needs -show_format:

cmd = "{0} ffprobe -loglevel {3} -print_format json -show_format -show_streams -select_streams v -i {1}

Then the next failure is here:

https://github.com/JNoDuq/videobench/blob/ea447031bfcc0315a4ae7d793896f051cc8bc2c3/videobench_functions.py#L375-L377

Terminal output logs deprecated API calls and advises using the newer ones, which are also documented here for ffmpeg vmaf. psnr=1 and model_path need to be changed:

    cmd = (''' {DOCKER_CMD} ffmpeg -y -loglevel {LOGLEVEL} -stats -i {CONTAINER_TMP_PATH}{REF_FILENAME} -i {CONTAINER_TMP_PATH}{INPUT_FILENAME} ''' \
        '''-lavfi "[0]{REF_DEINT}[refdeint];[refdeint]{REF_SCALE_FILTER}[ref];[1]setpts=PTS{SYNC_TIME}/TB[b];[b]{SCALE_FILTER}[c];[c][ref]libvmaf='n_threads={N_THREADS}:log_fmt=json:feature='name=psnr':model='path={VMAF_MODEL}':n_subsample={N_SUBSAMPLE}:log_path={CONTAINER_TMP_PATH}quality_{INPUT_NAME}.json'" ''' \
        ''' -t {DURATION} -f null - ''').format(

Now the quality_<input name>.json is properly generated..


Next failure is with psnr metrics. Presumably in the past this was a single value, inspecting the quality json file has related metrics listed like this:

"metrics": {
        "psnr_y": 57.247755,
        "psnr_cb": 57.478840,
        "psnr_cr": 57.988193,

These are values related to the individual channel components of the YCbCr colorspace, I don't know too much about this, and assume the previous single value according to the wikipedia article would have been the Y channel value. (EDIT: It is the correct one)

This should avoid the following error when an empty array is otherwise the current result:

input_obj.psnr_avg = round(sum(psnr_values)/len(psnr_values), 2)
ZeroDivisionError: division by zero

Relevant line:

https://github.com/JNoDuq/videobench/blob/ea447031bfcc0315a4ae7d793896f051cc8bc2c3/videobench.py#L248

Update to:

psnr_values.append(data_json['frames'][i]["metrics"]["psnr_y"]) ################################# get psnr values (libvmaf)

Now the processing completes successfully and actual graph results on the UI are drawn :)

150481878 commented 2 years ago

Same error. Have you solved that?

polarathene commented 2 years ago

Have you solved that?

Yes, I fixed all the issues locally to try this project.

You can see the devs profile is not very active on Github. Last activity was in Nov 2021, and Nov 2020 before that. They are active briefly about every 6-12 months it seems. I would choose a different project, or fork this one to maintain instead if it interests you.

Above your comment is another python project, that maintainer was responsive and active. They had similar breakage from the libvmaf 2.0 update, and I helped contribute the fixes there. I don't think it has some of the nice extras that this project does to pre-process videos to score correctly, and no GUI, just outputs PNG graphs, but it works well.

I have considered making a small project to read the JSON result data it outputs for interactive web browser graphs, but I do not know when/if I will have time for that. If you are on Windows there are other GUI projects around if that interests you, if you just want the metrics, CrypticSignal provides a good project that works :+1:

polarathene commented 2 years ago

@150481878 I noticed that I didn't quite explain how to resolve the final error clearly, I updated that comment for you.

I have checked the changes I made, nothing different from what I described above. I decided to create two PRs for them, you can see them both linked above this comment :)

150481878 commented 2 years ago

@150481878 I noticed that I didn't quite explain how to resolve the final error clearly, I updated that comment for you.

I have checked the changes I made, nothing different from what I described above. I decided to create two PRs for them, you can see them both linked above this comment :)

I noticed you only change 2 python files. I download them and replaced the old ones. It seems error still there. Here is my error log:


STARTING VIDEO BENCH python3 /home/lym/apps/videobench/videobench.py -ref /home/lym/crf_16.0.h264 -i /home/lym/crf_18.0.h264 -sync 0 -sw 0.0 -deint auto -subsampling auto -scale neighbor -vmaf_model auto -loglevel quiet


-> crf_16.0.h264 Bitrate : 24.82 Mbps avg_framerate : 24.0 interlaced : 0

Traceback (most recent call last): File "/home/lym/apps/videobench/videobench.py", line 183, in list_input_obj = manage_input_files(all_input, loglevel) File "/home/lym/apps/videobench/videobench.py", line 83, in manage_input_files input_obj.duration = float(ffprobe_json['streams'][0]['duration']) KeyError: 'duration'

polarathene commented 2 years ago

I noticed you only change 2 python files. I download them and replaced the old ones. It seems error still there.

Yes, but two different PRs. One fixes duration issue, the other one fixes the API update.

From that error, you're not using the duration fixes. If you are not a developer and familiar with PR diffs, let me know and I'll explain better. All changes required are detailed in the comment above though.

150481878 commented 2 years ago

I noticed you only change 2 python files. I download them and replaced the old ones. It seems error still there.

Yes, but two different PRs. One fixes duration issue, the other one fixes the API update.

From that error, you're not using the duration fixes. If you are not a developer and familiar with PR diffs, let me know and I'll explain better. All changes required are detailed in the comment above though.

I use the duration PR this time and replaced the old ones. Now the process went to 80% and hault. Maybe some new errors. Here is the log:


STARTING VIDEO BENCH python3 /home/lym/apps/videobench/videobench.py -ref /home/lym/crf_16.0.h264 -i /home/lym/crf_18.0.h264 -sync 0 -sw 0.0 -deint auto -subsampling auto -scale neighbor -vmaf_model auto -loglevel quiet


-> crf_16.0.h264 Bitrate : 24.82 Mbps avg_framerate : 24.0 interlaced : 0

-> crf_18.0.h264 Bitrate : 14.95 Mbps avg_framerate : 24.0 interlaced : 0

-> crf_18.0.h264 Reference deint_filter : null VMAF Model : /usr/local/share/model/vmaf_float_v0.6.1.json Scale filter : scale=1920:1080:flags=neighbor Quality Subsampling : 1

Calculate VMAF & PSNR (libvmaf - 16 treads)

docker container run --rm -v /tmp/videobench/:/home/shared-vmaf/ docker-videobench ffmpeg -y -loglevel quiet -stats -i /home/shared-vmaf/crf_16.0.h264 -i /home/shared-vmaf/crf_18.0.h264 -lavfi "[0]null[refdeint];[refdeint]scale=1920:1080:flags=neighbor[ref];[1]setpts=PTS+0.0/TB[b];[b]scale=1920:1080:flags=neighbor[c];[c][ref]libvmaf='n_threads=16:log_fmt=json:psnr=1:model_path=/usr/local/share/model/vmaf_float_v0.6.1.json:n_subsample=1:log_path=/home/shared-vmaf/quality_crf_18.0.json'" -t 155.155 -f null -

Traceback (most recent call last): File "/home/lym/apps/videobench/videobench.py", line 231, in datajson = json.load(open("{0}quality{1}.json".format( tmp_path , input_obj.name))) FileNotFoundError: [Errno 2] No such file or directory: '/tmp/videobench/quality_crf_18.0.json'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/lym/apps/videobench/videobench.py", line 233, in with open("{0}quality_{1}.json".format( tmp_path , input_obj.name), 'r') as file : #################### replace nan by 0 FileNotFoundError: [Errno 2] No such file or directory: '/tmp/videobench/quality_crf_18.0.json'

polarathene commented 2 years ago

Do you have files at /tmp/videobench/?

The failure is happening here:

https://github.com/JNoDuq/videobench/blob/ea447031bfcc0315a4ae7d793896f051cc8bc2c3/videobench.py#L224-L236

But I do not know what the cause would be, it seems to be specific to your setup. I probably cannot help you with resolving that further, have you considered trying https://github.com/CrypticSignal/video-quality-metrics ? It is simpler, but works well :)

150481878 commented 2 years ago

Do you have files at /tmp/videobench/?

The failure is happening here:

https://github.com/JNoDuq/videobench/blob/ea447031bfcc0315a4ae7d793896f051cc8bc2c3/videobench.py#L224-L236

But I do not know what the cause would be, it seems to be specific to your setup. I probably cannot help you with resolving that further, have you considered trying https://github.com/CrypticSignal/video-quality-metrics ? It is simpler, but works well :)

Thanks a lot. I will try that one.