VVC/H.266 support - Githubissues

rafael2k commented 3 years ago

Hi, Do you think it is in the scope of libheif the support for VVC/H.266 still image format? There is already a pretty decent free software implementation of VVC here: https://github.com/fraunhoferhhi/vvenc https://github.com/fraunhoferhhi/vvdec/

farindk commented 3 years ago

Hi, sure. It should be pretty easy to add VVC as encoder/decoder plugin to libheif. I don't have the HEIF DAmd2 text yet, so I don't know the prospective ID values for VVC. Once I have that information, VVC can be added.

rafael2k commented 3 years ago

Thanks! Can you point me the exact ISO standard you are mentioning? I can try to get through the university.

farindk commented 3 years ago

ISO/IEC 23008-12:2017/DAmd 2 Information technology — High efficiency coding and media delivery in heterogeneous environments — Part 12: Image File Format — Amendment 2: Support for VVC, EVC, slideshows and other improvements

https://www.iso.org/standard/81688.html

rafael2k commented 3 years ago

I dropped you an email. Please confirm. ; )

Jamaika1 commented 3 years ago

I was unsuccessfully trying in libbpg. https://github.com/Jamaika1/libbpg_vvc Reduce Complexity for intra coding of VVC https://github.com/llsurreal919/Reduce-Complexity-for-intra-coding-of-VVC The evc codecs are here. https://github.com/mpeg5/xeve https://github.com/mpeg5/xevd Currently, I advise against using the fraunhoferhhi codecs. I only support certain frame sizes 1280x720, 1920x1080 VVC codec can already be used in MP4 GPAC container. Strange that not in the MP5.

rafael2k commented 3 years ago

Hi Jamaika1,

It seems you suggested the use of the ISO reference code (in both "libbpg_vvc" and "Reduce-Complexity-for-intra-coding-of-VVC"), which is way slower than FhG implementation (and of course have some extra features). IMHO, FhG VVC are the best VVC open-source codec implementations, by far.

Jamaika1 commented 3 years ago

Hi rafael2k

FhG VVC v1.0.0 doesn`t currently have lossless function. It's still long way.

rafael2k commented 3 years ago

The purpose of such encoders are to have high compression rate, not to be bit-per-bit lossless. You are missing the point - higher coding efficiency at lower rates is the focus of these series of encoders (H.261, H.262, H.263, H.264, H.265, H.266 - and twin MPEG counter-parts). Indeed, at higher rates you get "lossless" compression considering the human visual system (HVS) - as you can verify in the psychovisual experiments in the literature. If you want to understand more about the FhG VVC codec, please read: J. Brandenburg, A. Wieckowski, T. Hinz , A. Henkel , V. George , I. Zupancic, C. Stoffers, B. Bross, H. Schwarz, D. Marpe, “Towards Fast and Efficient VVC Encoding,” IEEE 22nd Workshop on Multimedia Signal Processing (MMSP 2020), Tampere, Finland, 2020. and: A. Wieckowski, G. Hege, C. Bartnik, C. Lehmann, C. Stoffers, B. Bross, D. Marpe, “Towards a Live Software Decoder Implementation for the Upcoming Versatile Video Coding (VVC) Codec,” IEEE International Conference on Image Processing (ICIP 2020), Virtual Conference, 2020.

farindk commented 3 years ago

Both is correct. Of course, usually the lossy codec is used, but since h.264, there is also a lossless coding mode, which can actually be quite handy has a replacement alternative for PNG.

rafael2k commented 2 years ago

I'm impressed by EVC (thanks @Jamaika1), which performs a bit worse than VVC, but better then HEIF and AVIF, so I'm changing the title to VVC and EVC support.

Jamaika1 commented 1 year ago

I recommend GPAC HEIF link from VVC. MP4Box -add-image "image_21447_24bit.266":time=-1:@@reframer:frames=1 -ab heic "image.heic"

Track Importing VVC - Width 1563 Height 1558 FPS 25000/1000
VVC Import results: 1 samples (5 NALUs) - Slices: 1 I 0 P 0 B - 0 SEI - 1 IDR - 0 CRA
Adding sample at time 0.000 as item 3

Unfortunately. Demuxer isn't B-frame decompression. I am added link to nokia heif vvc. Unfortunately, I don't know how to use it. The decoder should already be vvdec thread and not jvet. https://github.com/nokiatech/heif/tree/VVC_MP4

rafael2k commented 1 year ago

I recommend GPAC HEIF link from VVC. MP4Box -add-image "image_21447_24bit.266":time=-1:@@reframer:frames=1 -ab heic "image.heic"
Track Importing VVC - Width 1563 Height 1558 FPS 25000/1000
VVC Import results: 1 samples (5 NALUs) - Slices: 1 I 0 P 0 B - 0 SEI - 1 IDR - 0 CRA
Adding sample at time 0.000 as item 3
Unfortunately. Demuxer isn't B-frame decompression. I am added link to nokia heif vvc. Unfortunately, I don't know how to use it. The decoder should already be vvdec thread and not jvet. https://github.com/nokiatech/heif/tree/VVC_MP4

Unfortunately, this implementation uses the VTM reference VVC for decoding, which is pretty buggy and slow. :(

rafael2k commented 1 year ago

Any news on VVC support?

Jamaika1 commented 1 year ago

Maybe use this: https://github.com/ffvvc/FFmpeg/commits/main

farindk commented 1 year ago

The main blocker for this at the moment is that in the current code, I would have to copy-paste a large part of it for each new compression format. I'd like to refactor this first so that there is less copied code.

farindk commented 1 year ago

Are there any example images using VVC? Adding the decoder would be a first step.

Jamaika1 commented 1 year ago

Hi farindk Here is example of heif VVC. https://www.sendspace.com/file/kuiwvw

farindk commented 1 year ago

@Jamaika1 Thank you. That will be very helpful.

rafael2k commented 1 year ago

That support would be really wonderful. Any help I can provide, let me know.

farindk commented 1 year ago

I started with the decoder. @Jamaika1 The example image you prepared has no "pitm" box. Would it be easy for you to add this? Otherwise, I have to patch it in a hex editor somehow.

farindk commented 1 year ago

The example image does not look correct to me. I started implementing decoding the vvcC box, but what I get does not match the specification:

| | | Box: vvcC -----
| | | size: 124   (header size: 8)
| | | version: 0
| | | frame-rate: 0
| | | constant frame rate: unknown
| | | num temporal layers: 0
| | | length size: 1
| | | chroma-format: 4:4:4
| | | bit-depth: 15

version should definitely by 1. I checked the file in a hex editor and skipped some bytes to that version=1, but then, everything else (bit-depth, chroma-format) is still incorrect. Moreover, the box is either much too long, or it contains config NALs, in which case, `num of arrays' should be >0.

farindk commented 1 day ago

There is now an experimental VVC encoding plugin in the master branch. We don't have any way to decode those images yet, though. Thus, I'm not sure it is written correctly.