Wrong ImageReturnProcessor results

[DanielProtopopov]

We scan two sides of the horizontal ID, first one is done through DocumentFaceRecognizer:

detectorRecognizer = new DocumentFaceRecognizer();
((DocumentFaceRecognizer) detectorRecognizer).setReturnFullDocumentImage(true);
((DocumentFaceRecognizer) detectorRecognizer).setReturnFaceImage(true);
((DocumentFaceRecognizer) detectorRecognizer).setNumStableDetectionsThreshold(3);
((DocumentFaceRecognizer) detectorRecognizer).setFaceImageDpi(400);
((DocumentFaceRecognizer) detectorRecognizer).setFullDocumentImageDpi(400);

After results come through OnScanningDone I re-create recognizer bundle and create scan for the back:

BarcodeRecognizer barcodeRecognizer = new BarcodeRecognizer();
barcodeRecognizer.setScanPdf417(true);
DocumentDetector cardDetector = buildDocumentDetectorFromPreset(DocumentSpecificationPreset.DOCUMENT_SPECIFICATION_PRESET_ID1_CARD);
DetectorRecognizer cardRecognizer = new DetectorRecognizer(cardDetector);

mBackImageReturnProcessor = new ImageReturnProcessor();
ProcessorGroup processorGroup = new ProcessorGroup(
     new Rectangle(-0.25f, -0.25f, 2.0f, 2.0f),
     new DPIBasedDewarpPolicy(400),
     mBackImageReturnProcessor
);
TemplatingClass documentClass = new TemplatingClass();
documentClass.setClassificationProcessorGroups(processorGroup);
cardRecognizer.setTemplatingClasses(documentClass);

Code for retrieving front scan results:

Recognizer firstRecognizer = Bundle.getRecognizers()[0];
DocumentFaceRecognizer documentFaceRecognizer = (DocumentFaceRecognizer) firstRecognizer;
if (documentFaceRecognizer.getResult().getResultState() == Recognizer.Result.State.Valid) {
mFrontDocumentImage = documentFaceRecognizer.getResult().getFullDocumentImage().clone();
mFaceImage = documentFaceRecognizer.getResult().getFaceImage().clone();

Toast.makeText(this, "Front of horizontal ID recognized", Toast.LENGTH_SHORT).show();
soundNotification();

mRecognizerView.resetRecognitionState();
mRecognizerView.pauseScanning();
mHaveResult = false;
ScanningFront = !ScanningFront;
Bundle = new RecognizerBundle(createRecognizers(ScanningFront, mOrientation));
Bundle.setAllowMultipleScanResultsOnSingleImage(true);
mRecognizerView.reconfigureRecognizers(Bundle);
mBtnToggleOrientation.setVisibility(View.INVISIBLE);
mRecognizerView.resumeScanning(true);

Code for the back scan results:

Recognizer recognizer = Bundle.getRecognizers()[0];
DetectorRecognizer detectorRecognizer = (DetectorRecognizer) recognizer;
if (detectorRecognizer.getResult().getResultState() == Recognizer.Result.State.Valid) {
if (Bundle.getRecognizers().length > 1) {
Recognizer firstRecognizer = Bundle.getRecognizers()[1];
BarcodeRecognizer barcodeRecognizer = (BarcodeRecognizer) firstRecognizer;
if (barcodeRecognizer.getResult().getStringData().length() != 0) {
Toast.makeText(this, "Back of the ID recognized", Toast.LENGTH_SHORT).show();
soundNotification();

mBackDocumentImage = mBackImageReturnProcessor.getResult().getRawImage().clone();
mRecognizerView.resetRecognitionState();
mRecognizerView.pauseScanning();
mHaveResult = false;

byte[] BarcodeData = barcodeRecognizer.getResult().getStringData().getBytes();
getIntent().putExtra("Barcode", Base64.encodeToString(BarcodeData, Base64.DEFAULT));
mRecognizerView.resumeScanning(true);
setResult(Activity.RESULT_OK);
finish();
}
}
}

Once the results come through, SOMETIMES, for unknown reason, in the mBackImageReturnProcessor there is a result for the FRONT, not the back. That happens in about half the times and I don't know why it is so because I do not create an image processor for the DocumentFaceRecognizer.

Confirmed - it happens on the third time I start the app, first two scans of back and front save proper results.

[DanielProtopopov]

I tested it on creating another derived application from ImagesSample using the code to create recognizers:

detectorRecognizer.setAllowFlippedRecognition(true);
detectorRecognizer.setDetectGlare(true);
this.mImageReturnProcessor = new ImageReturnProcessor();
ProcessorGroup processorGroup = new ProcessorGroup(new Rectangle(0.0f, 0.0f, 1.0f, 1.0f), new DPIBasedDewarpPolicy(NoUpScalingDewarpPolicy.DEFAULT_MAX_ALLOWED_DEWARPED_HEIGHT), this.mImageReturnProcessor);
TemplatingClass documentClass = new TemplatingClass();
documentClass.setClassificationProcessorGroups(processorGroup);
detectorRecognizer.setTemplatingClasses(documentClass);
BarcodeRecognizer pdf417Recognizer = new BarcodeRecognizer();
pdf417Recognizer.setScanPdf417(true);
pdf417Recognizer.setAutoScaleDetection(true);
pdf417Recognizer.setScanInverse(true);
pdf417Recognizer.setScanUncertain(true);
this.recognizerBundle = new RecognizerBundle(detectorRecognizer, pdf417Recognizer);
DocumentUISettings documentUISettings = new DocumentUISettings(this.recognizerBundle);
documentUISettings.enableHighResSuccessFrameCapture(true);
this.recognizerBundle.setAllowMultipleScanResultsOnSingleImage(true);
ActivityRunner.startActivityForResult((Activity) this, (int) SCAN_ACTIVITY_BACK_CODE, documentUISettings);

In OnActivityResult, the mImageReturnProcessor does not have the image for document recognizer. When it's done for the front (without PDF417 recognizer in the bundle), the image processor DOES have an image. The trouble is, if I wrap the detector recognizer for scanning back in the success frame grabber, it does not grab it either.

[i1E]

Hi @DanielProtopopov,

The problem here is that when you activate multiple recognizers at the same time, scan activity returns result immediately after one of the recognizres has successfully read data. If you want to obtain results from the both recognizers in a single scan, you should implement your custom scan activity with desired behaviour.

Good example is BlinkID-CustomUISample where you can find source code of MyScanActivity.

In onScanningDone callback, you should finish the scanning activity and return results only if both recognizers have valid results - Recognizer.Result.State.Valid.

The trouble is, if I wrap the detector recognizer for scanning back in the success frame grabber, it does not grab it either.

If you wrap BarcodeRecognizer with SuccessFrameGrabber, you will get full camera frame of the back side. SuccessFrameGrabber has result only if the wrapped recognizer has successfully read its data.

[DanielProtopopov]

Thank you @i1E, I've came to same conclusions and separated scanning into different activities to avoid mixing.