search

acmiyaguchi / game.moonlight

Moonlight client for Kodi Games
18 stars 9 forks source link

Implement h264 video rendering for video streaming #3

Closed acmiyaguchi closed 9 years ago

acmiyaguchi commented 9 years ago

The function int decoder_renderer_submit_decode_unit(PDECODE_UNIT decodeUnit) needs to be implemented under game.moonlight/src/callbacks/VideoCallbacks.cpp.

The current plan of action is to implement software rendering with support for Linux. Implementing or porting something like the video rendering in moonlight-embedded will be attempted.

The decoded output will be be fed directly into the Game API callback CHelper_libKODI_game::VideoFrame. Implementation of this callback should lead to similar implementations for audio and input.

Future plans may involve involve getting access to h264 rendering within Kodi to reduce library loads.

cgutman commented 9 years ago

I had hoped that Kodi had some sort of general video decoding/rendering API that could be used across add-ons, video players, and such. This way you could avoid having to roll your own video renderer and get platform-specific hardware acceleration for free. Seems like it would be easier but I obviously haven't done as much investigation in this area of Kodi.

acmiyaguchi commented 9 years ago

@cgutman You're right, it would be much easier to have something like that in place. Its possible to add onto the current game api to possibly add support for h264 rendering, but it'll take a bit a bit of work under the hood. I'm thinking I'll focus on getting a semi-workable demo using an outside decoder in the meanwhile.

I think I have a general idea at what exactly I'm looking at now, so I'll lay out my plan (mostly for my own sake). I've had decoding working on linux for a day or two now using ffmpeg to decode the streams. I stole a little pgm routine from their examples page to test the output, and they look fine. I'm a bit worried about network reliability and whether or not I'm properly handling the I had to do a bit of poking around to find out that YUV420P is yet to be implemented in CHelper_libKODI_game::VideoFrame, but I can just as easily render rgb32 (or rgb8888, it took me a bit to figure out the cryptic names for the pixel formats) in ffmpeg.

Running moonlight-common-c inside kodi does seem to make the networking stuff a bit less unreliable, by my metrics of how many scary looking errors I get. This may or may not be less of a problem on ethernet, I still have to try. This will probably be the next thing on tackling, and figuring out how much control I have over this issue.

acmiyaguchi commented 9 years ago

Decoding the frames has been offloaded to Retroplayer and Kodi's own ffmpeg decoder, since libav and ffmpeg weren't playing nicely. Streaming seems to work relatively well now, from limited testing.