FFMediaToolkit is a cross-platform video decoder/encoder library for .NET that uses FFmpeg native libraries. It supports video frames extraction, reading stream metadata and creating videos from bitmaps in any format supported by FFmpeg.
Decoding from System.IO.Stream was implemented in #54, but I didn't test it for real-time decoding. You can try passing a NetworkStream to the MediaFile.Load(stream), but I'm not sure if it will work.
Does the library allows to decode video to PNG in Real-Time?