Starting VideoPlayer from libgdx render thread

[marekhalmo]

Hello, i use the VideoPlayer in a Screen. I do vPlayer = VideoPlayerCreator.createVideoPlayer();
vPlayer.play(...)

in the initialize method (method which is called before the screen switch)

If i do this - no events are called.. I figured out that this is due to the fact that there is no Looper on LibGdx render thread.

So i added a handler to initialize and start the player. But the rendering then crashes on vPlayer.render() because there is a different GL context on textures for the mediaplayer.

I also try to call Looper.prepare() on the render thread but i get error that only one looper can exist.

Can you please suggest how can i create a working video player on a render thread of LibGDX game/screen?

Thanx

[RBogie]

I'm assuming this is a problem on android, because of the Looper and the texture problems.

What kind of events do you mean? I don't think I'm fully understanding what you're trying to do. Gdx-video doesn't need any Looper.

I haven't used gdx-video in complicated threading setups yet. Currently I've only run it in test cases specifically set up to test the player. One example of these testcases can be found here.

[marekhalmo]

Hello, starting from your test code the create method is run on main worker thread, the render method is not.

The render method runs on different thread than on create and therefore it won't have the same looper which is necessaary to call all the onXXXX listeners.

You can try this by moving try { videoPlayer.play(Gdx.files.internal("data/testvideo.ogv")); } catch (FileNotFoundException e) { e.printStackTrace(); }

to render method..

I will check now and give you a full example..

[marekhalmo]

Sry did not want to close that

[marekhalmo]

You can try this code

package com.gtomee.audiospectrum;

import java.io.FileNotFoundException;

import com.badlogic.gdx.ApplicationListener; import com.badlogic.gdx.Game; import com.badlogic.gdx.Gdx; import com.badlogic.gdx.InputMultiplexer; import com.badlogic.gdx.InputProcessor; import com.badlogic.gdx.files.FileHandle; import com.badlogic.gdx.graphics.Color; import com.badlogic.gdx.graphics.GL20; import com.badlogic.gdx.graphics.Mesh; import com.badlogic.gdx.graphics.PerspectiveCamera; import com.badlogic.gdx.graphics.VertexAttributes.Usage; import com.badlogic.gdx.graphics.g2d.BitmapFont; import com.badlogic.gdx.graphics.g3d.Environment; import com.badlogic.gdx.graphics.g3d.Material; import com.badlogic.gdx.graphics.g3d.Model; import com.badlogic.gdx.graphics.g3d.ModelBatch; import com.badlogic.gdx.graphics.g3d.ModelInstance; import com.badlogic.gdx.graphics.g3d.attributes.ColorAttribute; import com.badlogic.gdx.graphics.g3d.environment.DirectionalLight; import com.badlogic.gdx.graphics.g3d.utils.CameraInputController; import com.badlogic.gdx.graphics.g3d.utils.DefaultShaderProvider; import com.badlogic.gdx.graphics.g3d.utils.MeshBuilder; import com.badlogic.gdx.graphics.g3d.utils.MeshPartBuilder.VertexInfo; import com.badlogic.gdx.graphics.g3d.utils.ModelBuilder; import com.badlogic.gdx.math.Vector3; import com.badlogic.gdx.scenes.scene2d.Stage; import com.badlogic.gdx.scenes.scene2d.ui.Label; import com.badlogic.gdx.scenes.scene2d.ui.Table; import com.badlogic.gdx.scenes.scene2d.ui.Label.LabelStyle; import com.badlogic.gdx.utils.viewport.ScreenViewport; import com.badlogic.gdx.video.VideoPlayer; import com.badlogic.gdx.video.VideoPlayerCreator;

public class VideoTest extends Game implements ApplicationListener, InputProcessor { public PerspectiveCamera cam; public CameraInputController inputController; public ModelInstance instance; public Environment environment;

public VideoPlayer videoPlayer;
public Mesh mesh;

Stage stage;

@Override
public void create () {
    stage = new Stage(new ScreenViewport());
    stage.getViewport().update(Gdx.graphics.getWidth(), Gdx.graphics.getHeight(), true);

    LabelStyle lstyle =  new LabelStyle(new BitmapFont(), Color.WHITE);

    Label l = new Label("Stage is here!", lstyle);
    Table t = new Table();
    t.add(l).expand().fill();
    t.setFillParent(true);

    stage.addActor(t);

    videoPlayer = VideoPlayerCreator.createVideoPlayer();//cam, mesh, GL20.GL_TRIANGLES);
    videoPlayer.resize(Gdx.graphics.getWidth(), Gdx.graphics.getHeight());

    Gdx.gl.glEnable(GL20.GL_CULL_FACE);
    Gdx.gl.glCullFace(GL20.GL_BACK);
}
private boolean initialized = false;

@Override
public void render () {
    if(!initialized) {
        try {
            FileHandle fh = Gdx.files.external("data/testvideo.ogv");
            Gdx.app.log("TEST", "Loading file : " + fh.file().getAbsolutePath());
            videoPlayer.play(fh);
        } catch (FileNotFoundException e) {
            Gdx.app.log("TEST", "Err: " + e);
        }       
    }

    Gdx.gl.glClear(GL20.GL_COLOR_BUFFER_BIT | GL20.GL_DEPTH_BUFFER_BIT);

    if (!videoPlayer.render()) { // As soon as the video is finished, we start the file again using the same player.
        try {
            videoPlayer.play(Gdx.files.internal("data/testvideo.ogv"));
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        }
    }

    //stage.getBatch().begin();
    stage.act(Gdx.graphics.getDeltaTime());
    stage.draw();
    //stage.getBatch().end();
}

@Override
public void dispose () {
}

public boolean needsGL20 () {
    return true;
}

public void resume () {
}

public void resize (int width, int height) {
    if(stage.getWidth() != width || stage.getHeight() != height)
        stage.getViewport().update(width, height, true);

    if(videoPlayer != null)
        videoPlayer.resize(width, height);
}

public void pause () {
}

@Override
public boolean keyDown(int keycode) {
    // TODO Auto-generated method stub
    return false;
}

@Override
public boolean keyUp(int keycode) {
    // TODO Auto-generated method stub
    return false;
}

@Override
public boolean keyTyped(char character) {
    // TODO Auto-generated method stub
    return false;
}

@Override
public boolean touchDown(int screenX, int screenY, int pointer, int button) {
    // TODO Auto-generated method stub
    return false;
}

@Override
public boolean touchUp(int screenX, int screenY, int pointer, int button) {
    // TODO Auto-generated method stub
    return false;
}

@Override
public boolean touchDragged(int screenX, int screenY, int pointer) {
    // TODO Auto-generated method stub
    return false;
}

@Override
public boolean mouseMoved(int screenX, int screenY) {
    // TODO Auto-generated method stub
    return false;
}

@Override
public boolean scrolled(int amount) {
    // TODO Auto-generated method stub
    return false;
}

}

[marekhalmo]

If you put a breakpoint in the OnPreparedListener of VideoPlayerAndroid.java - it won't get called because there is no Looper running the render thread..

[RBogie]

Indeed I've only ran the example I gave you on the desktop. I'll have to figur out why my testcases for android were working fine. The documentation indeed states that the mediaplayer needs a Looper in order to function correctly.

I guess it would be best if the videoplayer handles this internally. However, introducing a new Looper thread only for the videoplayer seems wastefull. I'll have to investigate whether it is possible to get the android activity in order to run the creation of the videoplayer on the UI thread. I'm affraid that the activity will have to be changed for this to be possible, which means that creating a Looper thread is the only way.

I'll be investigating...

[marekhalmo]

Hello,

i'm afraid running the play routine on ui thread won't work (i tried and failed but give it a try your self, maybee i had a bug somewhere)

This is because you would use GL context of the texture in two different threads and that will give you an exception.. i had something like this:

02-18 16:28:14.868: W/GLConsumer(10644): [unnamed-10644-0] bindTextureImage: clearing GL error: 0x502 02-18 16:28:14.868: W/Adreno-ES20(10644): : GL_INVALID_OPERATION 02-18 16:28:14.868: E/GLConsumer(10644): [unnamed-10644-0] bindTextureImage: error binding external texture image 0x2: 0x502 02-18 16:28:14.898: W/dalvikvm(10644): threadid=14: thread exiting with uncaught exception (group=0x41926da0) 02-18 16:28:14.908: E/AndroidRuntime(10644): FATAL EXCEPTION: GLThread 27422 02-18 16:28:14.908: E/AndroidRuntime(10644): Process: sk.maniacs.bm, PID: 10644 02-18 16:28:14.908: E/AndroidRuntime(10644): java.lang.RuntimeException: Error during updateTexImage (see logcat for details) 02-18 16:28:14.908: E/AndroidRuntime(10644): at android.graphics.SurfaceTexture.nativeUpdateTexImage(Native Method) 02-18 16:28:14.908: E/AndroidRuntime(10644): at android.graphics.SurfaceTexture.updateTexImage(SurfaceTexture.java:169) 02-18 16:28:14.908: E/AndroidRuntime(10644): at com.badlogic.gdx.video.VideoPlayerAndroid.render(VideoPlayerAndroid.java:221)

[RBogie]

Internally I can leave the texture creation on the render thread. I would then only create the media player on the mail Looper thread.

When I have the time to try this, I'll let you know whether this works.

[marekhalmo]

Thanx.. your support is greatly appreciated!

[marekhalmo]

Hello, any news?

[RBogie]

Unfortunately I haven't found any time yet to try it out. I'll let you know when I do.

[marekhalmo]

Ok. Thanx

[dingjibang]

come on!i'm looking forward this project

[marekhalmo]

Same here :) .. this gdx extension is epic win,.. would help me a lot!

[RBogie]

Yeah, I'm sorry this is taking so long. I've been very busy with my study, and at work. Unfortunately, this is an issue that is not solvable with just a few minutes. I'll try it as soon as possible.

[marekhalmo]

Hello, any news?

[RBogie]

Yeah, I've tested the video player specifically on android in a newly created application, which worked just fine. The player can be created from the create function in your application. The play() function can be called from both this thread and the rendering thread. All callbacks work as they should. I did find a bug with reusing a player, which I solved (The fix is in the develop branch).

The code I tried is the following:

    VideoPlayer videoPlayer;
    Stage stage;

    boolean videoLoaded = false;

    @Override
    public void create () {
        stage = new Stage(new ScreenViewport());
        stage.getViewport().update(Gdx.graphics.getWidth(), Gdx.graphics.getHeight(), true);

        Label.LabelStyle lstyle =  new Label.LabelStyle(new BitmapFont(), Color.WHITE);

        Label l = new Label("Stage is here!", lstyle);
        Table t = new Table();
        t.add(l).expand().fill();
        t.setFillParent(true);

        stage.addActor(t);

        videoPlayer = VideoPlayerCreator.createVideoPlayer();
        videoPlayer.resize(Gdx.graphics.getWidth(), Gdx.graphics.getHeight());
        videoPlayer.setOnVideoSizeListener(new VideoPlayer.VideoSizeListener() {
            @Override
            public void onVideoSize(float v, float v2) {
                videoLoaded = true;
            }
        });

        try {
            FileHandle fh = Gdx.files.internal("small.mp4");
            Gdx.app.log("TEST", "Loading file : " + fh.file().getAbsolutePath());
            videoPlayer.play(fh);
        } catch (FileNotFoundException e) {
            Gdx.app.log("TEST", "Err: " + e);
        }
    }

    @Override
    public void render () {
        Gdx.gl.glClearColor(0, 0, 0, 1);
        Gdx.gl.glClear(GL20.GL_COLOR_BUFFER_BIT);

        if(videoLoaded) {
            if (!videoPlayer.render()) { // As soon as the video is finished, we start the file again using the same player.
                try {
                    videoLoaded = false;
                    videoPlayer.play(Gdx.files.internal("small.mp4"));
                    Gdx.app.log("TEST", "Started new video");
                } catch (FileNotFoundException e) {
                    e.printStackTrace();
                }
            }
        }

        stage.act(Gdx.graphics.getDeltaTime());
        stage.draw();
    }

However, when checking whether this worked because the creation thread had a looper, I found out that there is no looper in any thread used for rendering or creating or whatever. I really don't know why the callbacks are working. Therefore, I intend to change the behaviour so that the videoplayer will get the application's main looper, and use that to create the videoplayer. It will complicate the internals of the android videoplayer a bit, but at least then we adhere to the docs.

This implementation however, will take me some more time.

[marekhalmo]

Hello,

Thanx for your answer. I will try this, but i think it still does not work with my usecase. What you are doing here is using play on Game.create() .. This is not what is expected in 99% of use cases that i can imagine with this lib

This is a very simplified usecase:

• Create app that has a single "play" Button on stage that plays a video
• The play button on click listener must call vplayer.play(FileHanlde) - this allows multiple videos to be played by different button clicks
• render method can't have the replay function as it could block the rendering thread and lead to lags

Now because the click event does not have a looper - the media player won't ever call the events so the video won't start.

[RBogie]

Indeed I use the play call in the create method. However, it can be used anywhere. It does not have any dependencies that it needs to get from a thread. Textures etc are managed from the render() call, which should always reside on the rendering thread anyway. I will implement the videplayer creation inside of a looper thread. This will fix issues one might have with the events.

However, I checked and in my example none of the threads have a looper registered. This means that apparantly on the samsung firmware of my phone (also tried with cyanogenmod), it does not depend on the thread having a looper.

I agree that calling the play function in the render thread is not the right thing to do. In android, it won't block so it would be ok. On the pc however, it does load quite a bit before returning. I feel like this is something that shouldn't be fixed inside of the extension. To solve it, I would have to add a thread which runs tasks inside of the videoplayer. If every extension would do this, there would be a lot of spilled memory on stacks etc. To solve it, you can use an AsyncExecuter in some thread, and create AsyncTasks yourself. This would also be usable in other parts of your application.

If replaying is something that is used often, I might add a setRepeat function to the videoplayer. This is easy for the android backend. However, the desktop would need quite some work, so this is something for the future.

I will try to implement the looper thingie this weekend :wink:

[RBogie]

I pushed a fix for the looper thread creation to the develop branch. The media player is now created on a looper thread. I also made playing files more efficient because calling play again does not create a new player anymore. It now reuses the old one.

[marekhalmo]

Thanx.. i will test this today..

[marekhalmo]

Hello, sorry i had a lot of work to do so i could not spend too much time on this. I can confirm that the development version is working now for all my use cases.

I would suggest to add the getPossition and setVolume method similar to what music has.

Thanx for your help and assistance.

M.

[RBogie]

Great. I'll close this issue then :smiley: Thank you for the suggestion!