search

watercolor-games / the-peacenet

Welcome to The Peacenet, a futuristic open-world hacking game built on the Peace Engine.
https://watercolorgames.net/
GNU General Public License v3.0
5 stars 1 forks source link

Use ffmpeg/libav for cutscene decode #12

Open DeclanHoare opened 6 years ago

DeclanHoare commented 6 years ago

PNV takes up a lot of disk space and is unusably slow to decode at 1080p. I'd like to store FMV cutscenes as OGV or MP4, but we'll need a good library for it.

I haven't been able to find any libre, Mono-compatible .NET wrappers for ffmpeg beyond the wide range of libraries that wrap the command line API and can't do much more than cropping and format conversion. This project https://github.com/mikeboers/PyAV is quite good. A good route might be to create a .NET equivalent using P/Invoke. Just using PyAV itself via IronPython or Python.NET might also be possible, but seems a tad 'out there'.

ghost commented 6 years ago

What if we waited for MonoGame 3.7 Stable? They're implementing native FMV support in that. May as well let the pros deal with that since they're the ones actually writing MonoGame, instead of us trying to hack something together and wasting our time when native support is coming.

DeclanHoare commented 6 years ago

If it's going to be like the video player in XNA then I don't think it will support streaming over the Internet, since it uses the content pipeline.

ghost commented 6 years ago

Do we really need to stream cutscenes over the Internet? Honestly I prefer to have stuff like events and announcements displayed as text. And if we do need video, couldn't we just find a way to embed webkit and thus have a fullscreen YouTube embed?

ghost commented 6 years ago

Good news for streaming directly from a file. I was able to dramatically increase video decoding speed by storing the frame buffer as a uint[] rather than a Color[].

This means that the CPU doesn't have to look up where in memory each field of the Color struct is. It's just playing with good old 32-bit unsigned integers. So, technically, less RAM usage (both reading and writing and just storage usage).

Also, since we're playing directly with uints and PNV stores pixels as uints in the first place, we only need to apply the XOR operation 1 time per pixel rather than 3 or 4 times - a.k.a, 3x or 4x decoding speed increase. Also we don't need to cast the resulting uint to a byte, which means no need to truncate the integer, no CLR magic, etc. So more of an increase.

Next thing to do is audio, since we DO have a Vorbis decoder it might be our best bet for audio - just figure out how long each flick lasts in milliseconds and store a Vorbis segment of that length at the end of the frame's pixel data.

Also I'm thinking of tweaking the PNV frame format a bit so it doesn't store fully transparent pixels...it'll just TELL you if there are transparent pixels, and how many are coming up. So the encoder has to store less frame data (only the changes since the last frame) and the decoder can skip transparent pixels. Less disk space usage, and MOAR SPEEEEEEEEED.

DeclanHoare commented 6 years ago

That's the point of the XOR operation.

DeclanHoare commented 6 years ago

When a pixel is the same between frames (I'm assuming this is what you mean by "transparent") then the XOR produces 0. When many are the same in a row then the RLE will store a repeated zero.

If we make a special case for empty pixels so the decoder can skip the XOR, then you're introducing another branch in the decoder, which is more likely to make it slower. That's exactly what I had in mind when I designed the format in the first place.