search

OpenVisualCloud / SVT-HEVC

SVT HEVC encoder. Scalable Video Technology (SVT) is a software-based video coding technology that is highly optimized for Intel® Xeon® processors. Using the open source SVT-HEVC encoder, it is possible to spread video encoding processing across multiple Intel® Xeon® processors to achieve a real advantage of processing efficiency.
Other
507 stars 169 forks source link

Equirectangular projection ERP #605

Closed MisterNixon closed 3 years ago

MisterNixon commented 3 years ago

Hi, I and my team are working on a project and we want to be able to change the tile encoding to an ERP with SVT-HEVC on the OWT server client. We don't know exactly where to look to do some modifications in the source code. We are very new to video streaming. Our main goal is to be able to stream a 360-3d (stereo) video in 8k. FYI it's for a school project. Thank you!

tianjunwork commented 3 years ago

Hi @MisterNixon , not sure if you got your answer. SVT-HEVC can be used to encode either ERP or cubemap format 360 video. Tiling is a encoder feature. It has nothing to do with 360 video format. Let me know if you need help with encoder parameters.

MisterNixon commented 2 years ago

Thank you for your answer! Our goal is to be able to encode a 360 video using a Cubemap projection, connect a 360-3D 8k camera to the OWT-server to do a live streaming with a stereoscopic view (top-bottom). We have found this file (https://github.com/PFE-023-E2021/owt-server/blob/360-video/source/core/owt_base/SVTHEVCMCTSEncoder.cpp) in the owt-server / branch 360-video, we believe this file is responsible for making the tiles? Do you think we could enable a stereoscopic view (top-bottom) by editing this file, or it would be with some encoder parameter ? Because our camera diffuses a stereoscopic view (top-bottom).

If we understand properly, the projection does not matter to the encoder. The server simply selects the appropriate tiles according to the FOV and the projection, then the player generates the appropriate image. Can you confirm this theory?

Finally, if you could help us with encoder parameters to be able to encode with Cubemap instead of ERP, it would be very much appreciated!

Thank you!

tianjunwork commented 2 years ago

Hi @MisterNixon , a 360 video streaming pipeline is similar to a typical live streaming pipeline: camera -> on premise process/encode -> cloud ingestion -> decoder -> SVT-HEVC encoder -> origin -> CDN -> client Based on my knowledge, 360 video camera produces ERP format, it can be transformed to CMP on premise. Encoder(e.g. SVT-HEVC) on cloud get decoded CMP, then encode to HEVC bitstream using MCTS(using motion-constrained tile sets), which is needed by viewport-adaptive methods of 360 video streaming. Encoder simply encodes all the tiles. For FOV adaption, the client sends FOV information back to CDN (OWT-server), there is logic to handle tile selection. For tile-based encoder setting, set TileRowCount, TileColumnCount based on your CMP format, and UnrestrictedMotionVector: For MCTS support, set -umv 0. The reset is the same for regular encode.

MisterNixon commented 2 years ago

Hi @tianjunwork, we found a code line indicating the type of projection. On the other hand, we did change the parameter to E_SVIDEO_CUBEMAP by creating a fork of the project but the build failed as you can see here: https://github.com/PFE-023-E2021/owt-server/commit/29a6e7be91326bcee5ea38e03b6b2c65ab46e3db.

We are still looking for a way to enable a stereoscopic view. Could you help us with that? Thank you!

tianjunwork commented 2 years ago

Hi @MisterNixon , I am afraid your request is out of the scope of the support of this repo. If there is a business interest with 360 video streaming product, I can make the connection with our 360 video streaming team.