videos shoot on an iphone and rotated in the Photos app do not render any frames

[rjuszczyk]

The same video but not rotated plays just fine.

These are the logs for the rotated which doesnt render any video (just plays sound)

videotoolbox decoder: vt cvpx chroma: x420
transform filter error: Unsupported pixel size 0 (chroma CVPP)
main filter error: Failed to create video filter 'transform'
chain filter error: Too high level of recursion (3)
main filter error: Failed to create video converter
chain filter error: Too high level of recursion (3)
main filter error: Failed to create video converter
transform filter error: Format change is not allowed
main filter error: Failed to create video filter 'transform'
main vout display error: Failed to create video converter
main vout display error: Failed to adapt decoder format to display
main video output error: video output creation failed
main decoder error: failed to create video output

[rjuszczyk]

https://user-images.githubusercontent.com/1647827/200309772-56e5d02d-d1df-4a59-ba1b-727ecd952b36.mov

video which is having problems - originally shoot in landscape and then rotated in the Photos app to be portrait

[caprica]

Does it play in VLC?

[rjuszczyk]

Does it play in VLC?

Yes it does (sorry, I should have mentioned it)

[rjuszczyk]

Ironically it works fine on windows but still gives similar logs:

mmdevice audio output error: cannot initialize COM (error 0x80010106)
mmdevice audio output error: cannot initialize COM (error 0x80010106)
mmdevice audio output error: cannot initialize COM (error 0x80010106)
mmdevice audio output error: cannot initialize COM (error 0x80010106)
transform filter error: Unsupported pixel size 0 (chroma DX11)
transform filter error: Unsupported pixel size 0 (chroma DX11)
main filter error: Failed to create video filter 'transform'
transform filter error: Format change is not allowed
transform filter error: Format change is not allowed
main filter error: Failed to create video filter 'transform'
transform filter error: Unsupported pixel size 0 (chroma DX11)
transform filter error: Unsupported pixel size 0 (chroma DX11)
main filter error: Failed to create video filter 'transform'
transform filter error: Format change is not allowed
transform filter error: Format change is not allowed
main filter error: Failed to create video filter 'transform'
main vout display error: Failed to create video converter
main vout display error: Failed to adapt decoder format to display
main video output error: video output creation failed
main decoder error: failed to create video output
transform filter error: Unsupported pixel size 0 (chroma DXA9)
transform filter error: Unsupported pixel size 0 (chroma DXA9)
main filter error: Failed to create video filter 'transform'
transform filter error: Format change is not allowed
transform filter error: Format change is not allowed
main filter error: Failed to create video filter 'transform'
transform filter error: Unsupported pixel size 0 (chroma DXA9)
transform filter error: Unsupported pixel size 0 (chroma DXA9)
main filter error: Failed to create video filter 'transform'
transform filter error: Format change is not allowed
transform filter error: Format change is not allowed
main filter error: Failed to create video filter 'transform'
main vout display error: Failed to create video converter
main vout display error: Failed to adapt decoder format to display
main video output error: video output creation failed
main decoder error: failed to create video output
deprecated pixel format used, make sure you did set range correctly
main vout display error: Failed to set on top 

[caprica]

Well, the logs sometimes are not actually that important.

But in this case since VLC can play your video but the video output callbacks with LibVLC cannot - that difference is because different video outputs are used.

In the "normal" case, you have VLC render directly into a native window, or some OpenGL surface or whatever.

In the other case, software rendering, then it would appear that VLC cannot provide a converter from the source pixel/colour format to the target. The target is usually RV32, since that can be easily be rendered in Java (it's basically RGBA without A).

So, probably you need to find some picture format compatible with your source video that you can also render in Java.

Other than that, don't know.

[caprica]

To be honest, I wouldn't rely on some Photos app to robustly rotate a video, I'd use ffmpeg or something instead.

[rjuszczyk]

In my project I render it with CallbackVideoSurface and display(...) method of RenderCallback is never called. Does it rule out blaming video output callbacks not being able to play the video?

I cannot control how users are rotating their videos before importing it to my software.

[rjuszczyk]

Some findings so far:

running exiftool on this file shows

that it has:

Rotation                        : 270

so i tried removing the rotation metadata:

exiftool -rotation=0 video.mp4

that helps and now the video is played correctly (without rotation)

[caprica]

It is extremely unlikely anything can be done from the vlcj side here. I would recommend trying to use the EmbeddedMediaPlayer to see if it will play, as that is the most similar to what the VLC application itself does.

At least then you'll know that it simply won't work via the callback media player at all.

[caprica]

What happens if you run this:

import uk.co.caprica.vlcj.player.component.CallbackMediaPlayerComponent;
import uk.co.caprica.vlcj.player.component.EmbeddedMediaPlayerComponent;

import javax.swing.JFrame;
import java.awt.Point;

public class Test {

    public static void main(String[] args) throws InterruptedException {
        EmbeddedMediaPlayerComponent embedded = new EmbeddedMediaPlayerComponent();

        JFrame f1 = new JFrame("Embedded");
        f1.setContentPane(embedded);
        f1.setLocation(new Point(50, 100));
        f1.setSize(800, 600);
        f1.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        f1.setVisible(true);

        CallbackMediaPlayerComponent callback = new CallbackMediaPlayerComponent();

        JFrame f2 = new JFrame("Callback");
        f2.setContentPane(callback);
        f2.setLocation(new Point(900, 100));
        f2.setSize(800, 600);
        f2.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        f2.setVisible(true);

        String mrl = "CHANGE ME.mov"; // <--- change this

        embedded.mediaPlayer().media().play(mrl);
        callback.mediaPlayer().media().play(mrl);

        Thread.currentThread().join();
    }
}

[caprica]

With my JavaFX application, it also works.

This is on Linux, I don't know about macOS or Windows.

But, what this shows is that there is nothing wrong with vlcj here, your problem is somewhere else.

[caprica]

I see from your SO post that it was macOS it wasn't working on, well, there must be something wrong with the video output on macOS - whatever, the point is that vlcj is working and there is nothing that can be changed in vlcj to fix this issue for you.