Transcoding compression, video rotation -90 degrees, I expect no rotation after transcoding compression

[darwindu]

inputFile:

double displayRotation = grabber.getDisplayRotation();
displayRotation = -90.00

Causing video selection -90 degrees.

Can video transcoding compression not rotate -90 degrees by setting properties?

inputFile:

outputFile:

log：

2023-05-11 22:00:50,791 [main] INFO  JavacvUtils(JavacvUtils.java:82) - null 视频压缩：wight:1920, height:1080, format:mov,mp4,m4a,3gp,3g2,mj2, aspectRatio:1.0 displayRotation:-90.0 audioDisposition:0.0 videoDisposition:0.0
2023-05-11 22:00:56,056 [main] INFO  JavacvUtils(JavacvUtils.java:136) - null 转码完成
耗时(ms)：12141
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'D:\test\video.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2023-05-11T13:58:37.000000Z
    location        : +22.5332+113.9491/
    location-eng    : +22.5332+113.9491/
    com.android.version: 12
  Duration: 00:00:05.47, start: 0.000000, bitrate: 20608 kb/s
  Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, 19621 kb/s, 30.03 fps, 30 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2023-05-11T13:58:37.000000Z
      handler_name    : VideoHandle
      vendor_id       : [0][0][0][0]
    Side data:
      displaymatrix: rotation of -90.00 degrees
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 193 kb/s (default)
    Metadata:
      creation_time   : 2023-05-11T13:58:37.000000Z
      handler_name    : SoundHandle
      vendor_id       : [0][0][0][0]
[libopenh264 @ 000000002295c280] Slice count will be set automatically
[libopenh264 @ 000000002295c280] [OpenH264] this = 0x00000000043db9d0, Warning:layerId(0) doesn't support profile(578), change to UNSPECIFIC profile
[libopenh264 @ 000000002295c280] [OpenH264] this = 0x00000000043db9d0, Warning:bEnableFrameSkip = 0,bitrate can't be controlled for RC_QUALITY_MODE,RC_BITRATE_MODE and RC_TIMESTAMP_MODE without enabling skip frame.
Output #0, mp4, to 'D:\test\video_compress_javacv.mp4':
  Metadata:
    encoder         : Lavf59.27.100
  Stream #0:0: Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1920x1080, q=2-31, 2000 kb/s, 30 fps, 15360 tbn
    Metadata:
      rotate          : 90
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 193 kb/s
[aac @ 0000000022ade880] Qavg: 8073.756
[aac @ 0000000022ade880] 1 frames left in the queue on closing

Process finished with exit code 0

public static void compressFrame(File inputFile, File outputFile) throws Exception {
        String bizSeqNo = MumbleContextUtil.getBizSeqNo();
        FFmpegFrameGrabber grabber = FFmpegFrameGrabber.createDefault(inputFile);
        FFmpegFrameRecorder recorder = null;
        Frame captured_frame = null;
        try {
            grabber.start();
            int videoCodec = grabber.getVideoCodec();
            int width = grabber.getImageWidth();
            int height = grabber.getImageHeight();
            String format = grabber.getFormat();
            double aspectRatio = grabber.getAspectRatio();
            double displayRotation = grabber.getDisplayRotation();
            double audioDisposition = grabber.getAudioDisposition();
            double videoDisposition = grabber.getVideoDisposition();
            log.info("{} 视频压缩：wight:{}, height:{}, format:{}, aspectRatio:{} displayRotation:{} audioDisposition:{} videoDisposition:{}", bizSeqNo, width, height, format, aspectRatio, displayRotation, audioDisposition, videoDisposition);
            // audioChannels:设置重新编码的音频流中使用的声道数（1 =单声道，2 = 双声道（立体声））。如果未设置任何声道值，则编码器将选择默认值 0
            recorder = new FFmpegFrameRecorder(outputFile, width, height, grabber.getAudioChannels());
            // 视频帧率：帧率越低，效果越差；(保证视频质量的情况下最低25，低于25会出现闪屏)
            recorder.setFrameRate(30);
            // 采样率
            recorder.setSampleRate(grabber.getSampleRate());
            //设置视频压缩方式；视频编码属性配置 H.264 H.265 MPEG
            recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
            //设置视频比特率,单位:b
            recorder.setVideoBitrate(2000000);
            recorder.setVideoMetadata("rotate", "90");

            //recorder.setAspectRatio(aspectRatio);
            // yuv420p,像素
            recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);
            //设置视频质量，增加该参数，会导致视频增大
            //recorder.setVideoQuality(avutil.FF_LAMBDA_SHIFT);
            // 设置音频压缩方式：通用编码格式
            recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
            //设置音频比特率,单位:b (比特率越高，清晰度/音质越好，当然文件也就越大 128000 = 182kb)
            recorder.setAudioBitrate(grabber.getAudioBitrate());
            recorder.setAudioOptions(grabber.getAudioOptions());

            recorder.setVideoOption("preset", "ultrafast");
            recorder.start();
            while (true) {
                captured_frame = grabber.grabFrame();
                if (captured_frame == null) {
                    log.info("{} 转码完成", bizSeqNo);
                    break;
                }
                //recorder.setTimestamp(grabber.getTimestamp());
                recorder.record(captured_frame);
            }
            recorder.stop();
            recorder.release();
            grabber.stop();
            grabber.release();
        } catch (Exception e) {
            log.error("{} 压缩失败", bizSeqNo, e);
            throw e;
        } finally {

        }
    }

[saudet]

You'll need to call recorder.setVideoSideData(grabber.getVideoSideData()), but that property doesn't exist, yet, but contributions are welcome!

[saudet]

Duplicate of https://github.com/bytedeco/javacv/issues/1976

[darwindu]

Duplicate of #1976

Thank u.

recorder.setVideoSideData(grabber.getVideoSideData()) Is there a release plan for adding this attribute?

When can I call it?

[saudet]

Like I said, contributions are welcome. Anyone, include yourself, can create it!

[darwindu]

Like I said, contributions are welcome. Anyone, include yourself, can create it!

ok, thank u

[darwindu]

Like I said, contributions are welcome. Anyone, include yourself, can create it!

Supplementary explanation:

Property settings take effect recorder.setVideoMetadata("rotate", "90");

Tested versions 1.5.4 to 1.5.8

Version 1.5.4~1.5.6 is normal and will not rotate -90

Version 1.5.7~1.5.8 is abnormal and will rotate -90

Please note( the environment of the issues):

Windows rotates 90 degrees counterclockwise=-90

Linux rotates 90 degrees clockwise=90

[saudet]

Yes, the behavior of FFmpeg changed so that's not unexpected.

[saudet]

I've added setVideoSideData() and setDisplayRotation() in commit https://github.com/bytedeco/javacv/commit/b1c95f1fa838a9e3b11084095e11cb237682fbee. Please give it a try with the snapshots: http://bytedeco.org/builds/

[darwindu]

I've added setVideoSideData() and setDisplayRotation() in commit b1c95f1. Please give it a try with the snapshots: http://bytedeco.org/builds/

Sorry for the late reply,

Environment: win10 Version: javacv master feature

Three methods were tested to adjust the video direction:

1. recorder.setVideoSideData(grabber.getVideoSideData()); -- success
2. recorder.setDisplayRotation(270.0); -- success
3. recorder.setVideoMetadata("rotate", "90"); -- success

setVideoSideData log:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'D:\test\video.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2023-05-11T13:58:37.000000Z
    location        : +22.5332+113.9491/
    location-eng    : +22.5332+113.9491/
    com.android.version: 12
  Duration: 00:00:05.47, start: 0.000000, bitrate: 20608 kb/s
  Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, 19621 kb/s, 30.03 fps, 30 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2023-05-11T13:58:37.000000Z
      handler_name    : VideoHandle
      vendor_id       : [0][0][0][0]
    Side data:
      displaymatrix: rotation of -90.00 degrees
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 193 kb/s (default)
    Metadata:
      creation_time   : 2023-05-11T13:58:37.000000Z
      handler_name    : SoundHandle
      vendor_id       : [0][0][0][0]
[libopenh264 @ 0000000022420ac0] Slice count will be set automatically
[libopenh264 @ 0000000022420ac0] [OpenH264] this = 0x0000000003e35c00, Warning:layerId(0) doesn't support profile(578), change to UNSPECIFIC profile
[libopenh264 @ 0000000022420ac0] [OpenH264] this = 0x0000000003e35c00, Warning:bEnableFrameSkip = 0,bitrate can't be controlled for RC_QUALITY_MODE,RC_BITRATE_MODE and RC_TIMESTAMP_MODE without enabling skip frame.
Output #0, mp4, to 'D:\test\video_compress_javacv.mp4':
  Metadata:
    encoder         : Lavf59.27.100
  Stream #0:0: Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1920x1080, q=2-31, 2000 kb/s, 30 fps, 15360 tbn
    Side data:
      displaymatrix: rotation of -90.00 degrees
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 193 kb/s
[aac @ 00000000233ac1c0] Qavg: 8073.756
[aac @ 00000000233ac1c0] 1 frames left in the queue on closing

Process finished with exit code 0

setDisplayRotation log:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'D:\test\video.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2023-05-11T13:58:37.000000Z
    location        : +22.5332+113.9491/
    location-eng    : +22.5332+113.9491/
    com.android.version: 12
  Duration: 00:00:05.47, start: 0.000000, bitrate: 20608 kb/s
  Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, 19621 kb/s, 30.03 fps, 30 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2023-05-11T13:58:37.000000Z
      handler_name    : VideoHandle
      vendor_id       : [0][0][0][0]
    Side data:
      displaymatrix: rotation of -90.00 degrees
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 193 kb/s (default)
    Metadata:
      creation_time   : 2023-05-11T13:58:37.000000Z
      handler_name    : SoundHandle
      vendor_id       : [0][0][0][0]
[libopenh264 @ 0000000023490940] Slice count will be set automatically
[libopenh264 @ 0000000023490940] [OpenH264] this = 0x00000000237e0a30, Warning:layerId(0) doesn't support profile(578), change to UNSPECIFIC profile
[libopenh264 @ 0000000023490940] [OpenH264] this = 0x00000000237e0a30, Warning:bEnableFrameSkip = 0,bitrate can't be controlled for RC_QUALITY_MODE,RC_BITRATE_MODE and RC_TIMESTAMP_MODE without enabling skip frame.
Output #0, mp4, to 'D:\test\video_compress_javacv.mp4':
  Metadata:
    encoder         : Lavf59.27.100
  Stream #0:0: Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1920x1080, q=2-31, 2000 kb/s, 30 fps, 15360 tbn
    Side data:
      displaymatrix: rotation of -90.00 degrees
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 193 kb/s
[aac @ 0000000022e9df80] Qavg: 8073.756
[aac @ 0000000022e9df80] 1 frames left in the queue on closing

Process finished with exit code 0

setVideoMetadata log:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'D:\test\video.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2023-05-11T13:58:37.000000Z
    location        : +22.5332+113.9491/
    location-eng    : +22.5332+113.9491/
    com.android.version: 12
  Duration: 00:00:05.47, start: 0.000000, bitrate: 20608 kb/s
  Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, 19621 kb/s, 30.03 fps, 30 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2023-05-11T13:58:37.000000Z
      handler_name    : VideoHandle
      vendor_id       : [0][0][0][0]
    Side data:
      displaymatrix: rotation of -90.00 degrees
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 193 kb/s (default)
    Metadata:
      creation_time   : 2023-05-11T13:58:37.000000Z
      handler_name    : SoundHandle
      vendor_id       : [0][0][0][0]
[libopenh264 @ 000000002210e280] Slice count will be set automatically
[libopenh264 @ 000000002210e280] [OpenH264] this = 0x00000000229be2b0, Warning:layerId(0) doesn't support profile(578), change to UNSPECIFIC profile
[libopenh264 @ 000000002210e280] [OpenH264] this = 0x00000000229be2b0, Warning:bEnableFrameSkip = 0,bitrate can't be controlled for RC_QUALITY_MODE,RC_BITRATE_MODE and RC_TIMESTAMP_MODE without enabling skip frame.
Output #0, mp4, to 'D:\test\video_compress_javacv.mp4':
  Metadata:
    encoder         : Lavf59.27.100
  Stream #0:0: Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1920x1080, q=2-31, 2000 kb/s, 30 fps, 15360 tbn
    Metadata:
      rotate          : 90
    Side data:
      displaymatrix: rotation of -90.00 degrees
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 193 kb/s
[aac @ 000000002210ee00] Qavg: 8073.756
[aac @ 000000002210ee00] 1 frames left in the queue on closing

Process finished with exit code 0