Some ideas about improve the 4k video software decoding performance.

MouriNaruto commented 6 years ago

I think I should open an issue instead of reply to an issue. So I opened.

I have tested the @lukasf 's work in three days ago. But find its performance is same as the master branch's. So I have to try to optimize it by myself and I found something works.

First, the BufferTime property of MediaStreamSource should set to 0 to disable buffering. It can extremely significant reduce the memory usage. (From 10GB+ to 700MB, in my 4k 120fps avc 8 bit video.)

Second, I use the performance profiler in visual studio and find the DataWriter object in the UncompressedVideoSampleProvider class eats 10% of my cpu usage. So I have written a class to convert pointer to IBuffer directly.

class BufferReference : public RuntimeClass<
    RuntimeClassFlags<RuntimeClassType::WinRtClassicComMix>,
    abi_IBuffer,
    abi_IBufferByteAccess>
{
private:
    UINT32 m_Capacity;
    UINT32 m_Length;
    byte* m_Pointer;

public:
    virtual ~BufferReference() throw()
    {
    }

    STDMETHODIMP RuntimeClassInitialize(
        byte* Pointer, UINT32 Capacity) throw()
    {
        m_Capacity = Capacity;
        m_Length = Capacity;
        m_Pointer = Pointer;
        return S_OK;
    }

    // IBufferByteAccess::Buffer
    STDMETHODIMP Buffer(byte **value) throw()
    {
        *value = m_Pointer;
        return S_OK;
    }

    // IBuffer::get_Capacity
    STDMETHODIMP get_Capacity(UINT32 *value) throw()
    {
        *value = m_Capacity;
        return S_OK;
    }

    // IBuffer::get_Length
    STDMETHODIMP get_Length(UINT32 *value) throw()
    {
        *value = m_Length;
        return S_OK;
    }

    // IBuffer::put_Length
    STDMETHODIMP put_Length(UINT32 value) throw()
    {
        if (value > m_Capacity)
            return E_INVALIDARG;
        m_Length = value;
        return S_OK;
    }
};

// The M2MakeIBuffer function retrieves the IBuffer object from the provided 
// raw pointer.
//
// Parameters:
//
// Pointer
//     The raw pointer you want to retrieve the IBuffer object.
// Capacity
//     The size of raw pointer you want to retrieve the IBuffer object.
//
// Return value:
//
// If the function succeeds, the return value is the IBuffer object from the 
// provided raw pointer. If the function fails, the return value is nullptr.
//
// Warning: 
// The lifetime of the returned IBuffer object is controlled by the lifetime of
// the raw pointer that's passed to this method. When the raw pointer has been 
// released, the IBuffer object becomes invalid and must not be used.
IBuffer^ M2MakeIBuffer(byte* Pointer, UINT32 Capacity) throw()
{
    IBuffer^ buffer = nullptr;

    ComPtr<BufferReference> bufferReference;
    if (SUCCEEDED(MakeAndInitialize<BufferReference>(
        &bufferReference, Pointer, Capacity)))
    {
        buffer = reinterpret_cast<IBuffer^>(bufferReference.Get());
    }

    return buffer;
}

Then I found the maximum fps from 70 to 100. (The goal fps value is 120, just like PotPlayer on my machine.)

I hope someone find a better way to solve, this is the reason why I share it.

I hope it can help you.

Mouri