This code sometimes captures 4K image and sometimes only captures HD image. The canny based metric used in this code is much smaller on HD image than on 4K image (especially in night).
Therefore, I think it is necessary to check the size before calculating the metric. If it is not 4K, capture a new one. Besides, I think we can try to use bilateral filter for night image.
FRAME frame = grabMCamFrame(portbase+j, 1.0 );
int imgsize = frame.m_metadata.m_size;
size_t step=CV_AUTO_STEP;
Mat rawdata = Mat(1, imgsize , CV_8UC1, (void *)frame.m_image); //compressed jpg data
Mat loaded = imdecode(rawdata,1);
This code sometimes captures 4K image and sometimes only captures HD image. The canny based metric used in this code is much smaller on HD image than on 4K image (especially in night).
Therefore, I think it is necessary to check the size before calculating the metric. If it is not 4K, capture a new one. Besides, I think we can try to use bilateral filter for night image.