search

ViveSoftware / ViveMediaDecoder

A high performance video decoding Unity plugin for Windows.
http://u3d.as/usf
Other
129 stars 36 forks source link

Huge time difference between Native Script and c# decoding time #24

Open spaul13 opened 5 years ago

spaul13 commented 5 years ago

From the unity C# script ViveMediaDecoder.cs, I am getting the decoding time (time to change the state from BUFFERING to START is really high in the range of 60-90 ms but what I observed from the native script the video_decoding consists of 4 functions av_read_frame(), updatevideoframe(), av_packet_unref() and updateBufferState() which cumulatively takes no more than 25 ms and the updatevideoframe() takes the most of the time.

can anyone (@kyo8568131) please tell me why there is a large time gap between native script decoding time and decoding time observed from c# unity script?

kyo8568131 commented 5 years ago

Hi spaul13,

How large is your native buffer size? The state change time consists of the duration for filling native buffer and frame update interval time. If native buffer size is 4, that means you may take 4 frame decoding time plus the duration from native buffer full to Unity calling update and check the state. In worst case, the duration from buffer full to Unity calling update would be your screen refresh time, for example, 90 frames per second means 11.11 ms per frame.

spaul13 commented 5 years ago

mVideoBuffMax = 64 and SWAP_BUFFER_NUM = 4 for audio (i.e. my clip doesn't contain any audio, hence audio is disabled). I tried to check the timing by decreasing the mVideoBuffMax size defined in the native script but it's not changing the result. I found these two buffer sizes which are defined.

@kyo8568131 can u please tell is there any other buffer size which I can change to improve the performance? I got your idea but I wasn't able to find any other buffer size specifications. Any suggestions will be highly appreciated.

kyo8568131 commented 5 years ago

No, there is no other buffer. Please help to check the buffer size is not over written by config file. And profile also the Unity life cycle part for detail performance analysis.

spaul13 commented 5 years ago

@kyo8568131, I already checked it, the buffersize is not overwritten by anything. I will do profiling on unity but the decoding is performed asynchronously so I guess the observed effect of expected delay enhancement from profiling will be minimal.

Are u suggesting me to keep the mVideobuffMax size to a very low value? (2/4)

kyo8568131 commented 5 years ago

The buffer is to prevent if something makes decoding late. It should be OK for general case, I mean, the time of decode 1 frame is lesser than frame update.