Retrieve Speed

[toddhlynch]

I have the Raspberry Pi Cam 2 NoIR on a Raspberry Pi Zero, and I am successfully capturing frames with the following code, which is based heavily on the sample code provided in this repo.

I changed it to color format (CV_8UC3) and set the width and height to the maximum supported by video mode --- which is supposed to support 15 frames per second.

I would prefer to use the still capture mode, but I am ok using the video capture. I would like as high a resolution as possible.

The Camera.retrieve(image); step takes close to 3-4 seconds to execute, I would expect it to take under 100ms to support 15 fps.

Is there something obvious I am missing? Is it possible to have the camera waiting for still capture and trigger it quickly? Or is it better to have the video running in a continual loop like I have and simply take the next frame when I need it?

void CServer::roll() {

    raspicam::RaspiCam_Cv Camera;

    Camera.set( CV_CAP_PROP_FORMAT, CV_8UC3 );
    Camera.set( CV_CAP_PROP_FRAME_WIDTH, 2592 );
    Camera.set( CV_CAP_PROP_FRAME_HEIGHT, 1944 );
    Camera.set( CV_CAP_PROP_FPS, 15 );

    //Open camera
    cout<<"Opening Camera..."<<endl;
    if (!Camera.open()) {cerr<<"Error opening the camera"<<endl;return;}

    //wait a while until camera stabilizes
    sleep(3);
    cout<<"Camera ready..."<<endl;

    cv::Mat image;

    vector<int> compression_params;
    compression_params.push_back(CV_IMWRITE_PNG_COMPRESSION);
    compression_params.push_back(9);

    while(true) {
            cout<<"Camera Grab..."<<endl;
            Camera.grab();

            cout<<"Camera Retrieve to Mat..."<<endl;
            Camera.retrieve(image);

            cout<<"Image saved at image.png"<<endl;
            cv::imwrite("image.png", image, compression_params);

    }
}

[toddhlynch]

As a follow up. The retrieve function simply is:

/**
*Decodes and returns the grabbed video frame.
*/
void RaspiCam_Cv::retrieve ( cv::Mat& image ) {
//here we go!
    image.create ( _impl->getHeight(),_impl->getWidth(),imgFormat );
    _impl->retrieve ( image.ptr<uchar> ( 0 ));
}

Which creates a blank RGB image of the proper height and width and then _impl->retrieve calls...

        void Private_Impl::retrieve ( unsigned char *data,RASPICAM_FORMAT type ) {
            if ( callback_data._buffData.size==0 ) return;
            if ( type!=RASPICAM_FORMAT_IGNORE ) {
                cerr<<__FILE__<<":"<<__LINE__<<" :Private_Impl::retrieve type is not RASPICAM_FORMAT_IGNORE as it should be"<<endl;
            }
            memcpy ( data,callback_data._buffData.data,getImageTypeSize ( State.captureFtm ) );
        }

Which just copies the camera buffer data to the image Mat. Any ideas why it is taking so long?

[luntik2012]

did you measured execution time of every single step inside the lib? possibly, the bottleneck is in rpi zero gpu encoding speed. i have 28.5 fps on rpi 3b with rgb format