Unable to play http/rtmp live stream using vlcj

[rahulsharma1981]

Hi,

I am using vlcj 2.4.0 library on windows and trying to play http and rtmp Live stream. The stream starts playing but after some time it shows three behaviors randomly 1) The video frames stopped playing and audio is continued to play. 2) Video and audio both stopped playing.

Many times it reflects the video frames stopped playing and audio is continued to play.

I get following errors in my trace:

libdvbpsi error (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
libdvbpsi error (PSI decoder): TS duplicate (received 0, expected 1) for PID 4095
libdvbpsi error (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
libdvbpsi error (PSI decoder): TS duplicate (received 0, expected 1) for PID 4095
libdvbpsi error (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
libdvbpsi error (PSI decoder): TS duplicate (received 0, expected 1) for PID 4095
libdvbpsi error (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
libdvbpsi error (PSI decoder): TS duplicate (received 0, expected 1) for PID 4095
libdvbpsi error (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
libdvbpsi error (PSI decoder): TS duplicate (received 0, expected 1) for PID 4095

please help me.

Thanks & Regards Rahul K Sharma

[caprica]

Those are errors from the native LibVLC process, not vlcj. Remember that vlcj does not play your video, VLC does.

As to why it's happening, I have no idea. You should probably post a failing test case at least, but this is unlikely to be a vlcj error, more likely a problem with your sout string or transcoding.

Play your stream in the VLC application and see what happens.

[rahulsharma1981]

when I play stream using VLC it is playing smoothly. I cannot send the file more that 10 MB please suggest how can I send you the test application and sample video.

[caprica]

What is the MRL you are playing? What are the media options you are using?

[rahulsharma1981]

We are streaming a media file using WOWZA server, and playing the stream by the application build using vlcj. And the URL of the stream is http://192.168.1.103:1935/ISM/mp4:cache\hd.mp4/playlist.m3u8 If we play the file from our local HD location it is playing well but when we play the Wowza live stream it gives the problem.

[caprica]

Do you have access to the same stream (or something similar) that I could connect to over the internet to test this?

[rahulsharma1981]

I can send the media file to you. I do not have any public URL.

[caprica]

From what you said already, the media file is not the problem. The streaming is.

I am not going to set up my own WOWZA server to test this for you.

[rahulsharma1981]

Yes I understand, I find http://www.nasa.gov/multimedia/nasatv/NTV-Public-IPS.m3u8 a public URL which is showing same behavior and trace. May be this will be useful.

[caprica]

If I open that NASA-TV MRL inside the VLC application, I see these messages in the log (via Tools -> Messages):

ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 256
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 256
httplive warning: playback in danger of stalling
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 256
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 256
httplive warning: playback in danger of stalling
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 256
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 256
httplive warning: playback in danger of stalling
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 256

The video appears to play OK though.

[caprica]

How long after you start the stream do the problems (losing video) begin?

I am playing that NASA-TV stream in the vlcj TestPlayer just fine so far...

[rahulsharma1981]

same message coming to my end but after some time video frames being stopped only audio is coming. but with VLC it is playing well.

[caprica]

So the messages must be irrelevant.

How long is "some time".

I've been 'watching' this stream for ten minutes and I can't stand the tedium of watching it much longer...

[caprica]

Run the TestPlayer application from the vlcj sources, press the third button from the right (it looks like two cables being plugged together).

A dialog box appears with an input box. Type in the URL of your stream into the input box.

The stream should start playing.

Report observations.

[caprica]

This NASA-TV stream is still playing just fine for me in the vlcj test application, that's like 45 minutes or so. I can't watch it all day.

[rahulsharma1981]

you mean to say problem is something else not related to error log bellow

ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 256
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
ts error: libdvbpsi (PSI decoder): TS duplicate (received 0, expected 1) for PID 256
httplive warning: playback in danger of stalling

[caprica]

I mean to say that the errors in the log seem to have no bearing on the problem, since:

a) when you play the stream in VLC you tell me it works just fine, and when I play it in VLC it looks like it works just fine - but the 'error' messages still appear in the VLC log; b) I played that NASA-TV stream for nearly an hour with no apparent problems, despite the 'error' messages appearing in the log.

So I suspect something wrong with your application instead.

That's why I suggested that you try playing the stream in the vlcj TestPlayer application and report your observations. Because if that works, then certainly the error is in your own code (e.g. maybe something is getting garbage collected that shouldn't be).

So try the vlcj TestPlayer and report back...

And you still didn't tell me how long after you started the stream did the problems occur.

I can't really help you any more than I already have.

[rahulsharma1981]

OK will report back soon.

[rahulsharma1981]

Hi Mark,

I Investigated further and I find that my problem is related with http://stackoverflow.com/questions/13248643/playing-large-videos-fullhd-with-vlcj-using-directmediaplayer I am using following code for this:

public class PlayerMgr {

    private static final boolean useSourceSize = true;

    private final Canvas canvas;

    private final PixelWriter pixelWriter;

    private final WritablePixelFormat<ByteBuffer> pixelFormat;

    private final BorderPane borderPane;

//    private DirectMediaPlayerComponent mediaPlayerComponent = null;
    private DirectMediaPlayer mp = null;
    private Slider timeSlider;

    Timeline updateSliderLine = null;
    private final AtomicInteger frameNumber = new AtomicInteger();

    public PlayerMgr(BorderPane borderPaneLocal, Slider slider) {
        this.borderPane = borderPaneLocal;
        this.timeSlider = slider;

        canvas = new Canvas();
        pixelWriter = canvas.getGraphicsContext2D().getPixelWriter();
        pixelFormat = PixelFormat.getByteBgraPreInstance();

        borderPane.setCenter(canvas);
        MediaPlayerFactory factory = new MediaPlayerFactory("--ffmpeg-hw");

        mp = factory.newDirectMediaPlayer(
                new BufferFormatCallback() {
                    @Override
                    public BufferFormat getBufferFormat(final int width, final int height) {
                        System.out.println("[FINE] VLCPlayer: Buffer size changed to " + width + "x" + height);
                        double[] aspectRatioWidth = maintainAspectRatioOfImage(width, height, borderPane.getWidth(), borderPane.getHeight());
                        canvas.setWidth(aspectRatioWidth[0]);
                        canvas.setHeight(aspectRatioWidth[1]);
                        return new RV32BufferFormat((int) aspectRatioWidth[0], (int) aspectRatioWidth[1]);
                    }
                },
                new RenderCallback() {
                    AtomicReference<ByteBuffer> currentByteBuffer = new AtomicReference<>();
                    int counter = 0;
                    @Override
                    public void display(DirectMediaPlayer mp, Memory[] memory, final BufferFormat bufferFormat) {
                        final int renderFrameNumber = frameNumber.incrementAndGet();
                        currentByteBuffer.set(memory[0].getByteBuffer(0, memory[0].getSize()));
                        ByteBuffer byteBuffer = currentByteBuffer.get();
                        int actualFrameNumber = frameNumber.get();
                        if (renderFrameNumber == actualFrameNumber) {
                            pixelWriter.setPixels(0, 0, bufferFormat.getWidth(), bufferFormat.getHeight(), pixelFormat, byteBuffer, bufferFormat.getWidth() * 4);
                        }
                        System.out.println("The frames being populated are ......  " + (++counter));
                    }
                }
        );

//            mediaPlayerComponent = new PlayerMgr.TestMediaPlayerComponent();
        updateValues();
    }

    public void updateValues() {
        updateSliderLine = new Timeline(new KeyFrame(Duration.millis(100), new EventHandler<ActionEvent>() {
            @Override
            public void handle(ActionEvent arg0) {
                try {
                    long currentTime = mp.getTime();
                    long duration = mp.getLength();
                    if (duration > 0 && !timeSlider.isValueChanging() && currentTime > 0) {
                        double dur = (double) ((float) currentTime / (float) duration) * 100.0; //currentTime.divide(duration.toMillis()).toMillis() * 100.0;
                        timeSlider.setValue(dur);
                    }
                } catch (Exception ex) {
                    ex.printStackTrace();
                }
            }
        }));
        updateSliderLine.setCycleCount(Timeline.INDEFINITE);
        updateSliderLine.play();
    }
public void loadMedia(String videoPath) {
//        if(mp.isPlayable())
//            mp.release();

        mp.playMedia(videoPath);
    }

    public void play() {
        mp.play();
    }

    public void pause() {
        mp.pause();
    }

    public void stop() {
        mp.stop();
    }

    public void release() {
        if (mp != null && mp.isPlaying()) {
            mp.release();
        }
    }

    public double[] maintainAspectRatioOfImage(double imgWidth, double imgHgt, double maxWdth, double maxHgt) {
            double[] returnHgtWdt = new double[2];
            if (imgWidth > maxWdth) {
                double factorOfWidth = (imgWidth - maxWdth) / imgWidth;
                double encreaseHeight = imgHgt - (factorOfWidth * imgHgt);
                if (encreaseHeight > maxHgt) {
                    return maintainAspectRatioOfImage(maxWdth, encreaseHeight, maxWdth, maxHgt);
                }
                returnHgtWdt[0] = maxWdth;
                returnHgtWdt[1] = encreaseHeight;
            } else if (imgHgt > maxHgt) {
                double factoreHeight = (imgHgt - maxHgt) / imgHgt;
                double enceraseWidth = imgWidth - (factoreHeight * imgWidth);
                if (enceraseWidth > maxWdth) {
                    return maintainAspectRatioOfImage(enceraseWidth, maxHgt, maxWdth, maxHgt);
                }
                returnHgtWdt[0] = enceraseWidth;
                returnHgtWdt[1] = maxHgt;
            } else {
                returnHgtWdt[0] = imgWidth;
                returnHgtWdt[1] = imgHgt;
            }
            return returnHgtWdt;
        }
}

And I get following error when the frame updation being stopped:

[49c18364] httplive stream error: downloading segment 21 from stream 0 failed
[4f794474] main vout display error: Failed to change zoom
[4f794474] main vout display error: Failed to change source AR

Hope this will make my point clear and may be you will be able to suggest what I am doing wrong.

Thanks & Regards Rahul

[caprica]

Do you really need to use the "direct" rendering media player?

[rahulsharma1981]

My requirement is to implement vlcj in javaFx. I find example and samples with the use of DirectMediaPlayerComponent and DirectMediaPlayer.

[caprica]

Well, the implication in that StackOverflow answer that you linked to is that you have hit a finite performance limit - i.e. there is not enough physical bandwidth to play the video, and you need hardware acceleration. The "direct" media player renders in software.

[rahulsharma1981]

Hi Mark,

After further investigation I find that when I am using DirectMediaPlayer the memory uses graph increasing continuously. Then I nativeBuffer.clear(); in overrided display() method.It makes performance slightly better.

[caprica]

That's interesting, but I don't understand why it would make any difference (other than change timings that are obscuring a potential race condition in your code).

The reason I say that is that the javadoc for clear() states this:

"Zero the full extent of this memory region"

Why would that improve performance? It makes no sense.

[rahulsharma1981]

Mark, this memory issue is coming with my code as well as running with uk.co.caprica.vlcj.test.basic.TestPlayer.java and uk.co.caprica.vlcj.test.direct.DirectTestPlayer.java I find one more issue already been posted https://github.com/caprica/vlcj-javafx/issues/2.

[caprica]

OK, so if you think this is the same issue as the one you referenced, I'll close this here.

[caprica]

I would add that I have profiled the vlcj media player implementations for hours, and I have never seen a Java heap memory leak in my own vlcj applications.

[rahulsharma1981]

I can give you the sample code and sample video on which it can be replicated.How can I mail this to you? Or please give me the sample in which you are testing so that I can compare my code. I has tested this scenario with uk.co.caprica.vlcj.test.basic.TestPlayer.java as well as uk.co.caprica.vlcj.test.direct.DirectTestPlayer.java both are replicating this issue.

[caprica]

I just play any old video file I have hanging around, e.g. an FLV file from BBC iPlayer, or a DVD ISO image, or any old MP4.

Are you categorically saying that you have observed a Java heap memory leak with the unmodified vlcj TestPlayer? If so, I want to see a screendump of the jvisualvm heap charts (or some other profiler results) that you used to come to this conclusion.

I have not tested this recently, but it wasn't all that long ago, and I never saw any evidence of any Java heap memory leaking. So unless you, or someone else, provides evidence to the contrary, there is nothing new for me to spend my own time investigating.

Also, please make sure you are not conflating two separate issues into one - you have talked about two different things in this issue report: 1) an alleged memory leak; and 2) a performance issue (maybe due to limited CPU bandwidth).

[caprica]

Ten minutes, perfect GC cycling, no evidence whatsoever of any leak...

[caprica]

Twenty minutes, still no evidence of any heap memory leak...

[caprica]

Over half an hour... everything is still working correctly...

[caprica]

Based on the above heap charts, the only reasonable conclusion to draw is that there is, like I said earlier, no Java heap memory leak in the vlcj TestPlayer.

[rahulsharma1981]

Here is the memory chart you can see how memory increases.

[rahulsharma1981]

And Here is the Object that was actually using the memory.

[rahulsharma1981]

Here is the memory Graph as well.

[caprica]

That is not the vlcj TestPlayer is it. You said the vlcj TestPlayer showed the same problem.

Instead, that looks like a JavaFX player to me. You need to fully read both these issue reports, i.e. this one and the other one you referenced. In particular read the bit about PixelWriter.

Also, you need to get a heap dump and drill-down into it to see which objects (GC roots) are pinning those huge byte[] instances in memory and preventing them from being garbage collected.

[caprica]

After ten minutes running the vlcj JavaFXDirectRenderingTest class, my heap dump shows no such evidence of byte[] leaks, the approx 2Mb of byte[] data looks reasonable to me...

Are you absolutely sure you are running an unmodified vlcj test class? I thought earlier you said you were using some other code you found in a StackOverflow answer - if that's the case, I'm not responsible for it.