search

siralam / CameraViewPlus

Forked from Google's CameraView, but solved bugs and added features.
Apache License 2.0
86 stars 22 forks source link

Performance decreases drastically when set OnFrameListener #8

Closed oscarcpozas closed 5 years ago

oscarcpozas commented 6 years ago

I have noticed that when the listener is setted performance decreases (using camera2), probably for this conversion on all frames: latestFrameData = Utils.YUV420toNV21(image); It would be advisable to review this transformation.

siralam commented 6 years ago

@oscarcpozas Could you clarify more on "performace"? Do you mean the FPS of camera preview, or UI thread skipped frames?

I actually do latestFrameData = Utils.YUV420toNV21(image); on another thread separated with the camera; so I susepct, if you still encounter performance problem, I would say the device's CPU is not strong enough to support such feature....

With that said, I am very happy to see a faster way to convert YUV420 bytes to NV21.

oscarcpozas commented 6 years ago

I'm trying on Samsung Galaxy S8+ (SM-G955F), shouldn't go wrong. I'm not sure if it's skipping frames, but you can tell that the view is more lagging.

Since my ignorance, why not set to the FrameImageReader NV21 formar directly?

mFrameImageReader = ImageReader.newInstance(previewLargest.getWidth(), previewLargest.getHeight(),
                ImageFormat.NV21, 1)

Another question is, why are we using so many threads compared to the original google implementation?

siralam commented 6 years ago

@oscarcpozas I am not quite sure but it is possible that Samsung devices are sill using legacy camera, i.e. falling back to Camera1. To play safe, let's first check if it is really running on Camera2.

Since my ignorance, why not set to the FrameImageReader NV21 format directly?

I wish I could, you can just try it :( Caused by: java.lang.IllegalArgumentException: NV21 format is not supported Yeah wonderful Android Camera2 API

Another question is, why are we using so many threads compared to the original google implementation?

There are actually 2 more threads than the originally one: mFrameThread and mFrameProcessThread.

mFrameThread exists because of the onFrame feature, and therefore we need another ImageReader which has to be run in another thread separated with the camera preview;

mFrameProcessThread is not necessary if I am simply passing the raw bytes of Image to the user. However this is not a very good UX in my opinion; you see I have to find a way to convert YUV_420_888 to NV21 which is not something common user will know how to do. And I cannot do this on mFrameThread as well, because the mFrameThread is used for getting real-time frame images; if it is stuck because processing needs time, user will not get the most up-to-date image. So now when I separated them, once mFrameThread finds the previous frame is still processing, it can immediately drop the current image it is carrying and continuing to get the next image.

oscarcpozas commented 6 years ago

Sorry, for the delay in responding. I'm currently forcing the use of Camera 1, and I'll be back to this next week.

siralam commented 6 years ago

@oscarcpozas Wonder if you have any updates on this one~?