How to train it for other currencies?

[andro-galexy]

The code is awesome, am from India and I would like to know how i can train this repo , to recognise indian RUPEE with the same accuracy as that of for euro banknotes.

[carlosmccosta]

Good morning :)

For recognizing other currencies you need to update the image database and its metadata that is in this folder: https://github.com/carlosmccosta/Currency-Recognition/tree/master/CurrencyRecognition/imgs

The euro database uses 3 levels of resolution for improving recognition results. Also, each image has a bitmask that informs the system in which regions of the image it should look for keypoints. Finally, the list of train and test images is specified in these text files: https://github.com/carlosmccosta/Currency-Recognition/blob/master/CurrencyRecognition/imgs/listCurrencyDB.txt https://github.com/carlosmccosta/Currency-Recognition/blob/master/CurrencyRecognition/imgs/listTestDB.txt

In the listCurrencyDB.txt, each line has information about: filename | currency value | color for drawing banknote border in RGB

More info about database setup: https://github.com/carlosmccosta/Currency-Recognition/blob/master/CurrencyRecognition/src/ImageAnalysis/ImageDetector.cpp#L22

Have a nice day :)

[andro-galexy]

Thanks, for this issue, also am not good in c++, I know python better, so would it be feasible to convert this code in python.

[carlosmccosta]

Good morning,

I don't have time for porting this project to python. But it would be straight forward / easy task.

P.S. OpenCV is implemented in C++ because it offers much more efficient runtime code and is easy to maintain / evolve. As such, if you plan to use OpenCV i suggest you use C++. It's an easy language to learn and the time you would invest in learning it would pay off if you are implementing real time systems.

Have a nice day :)

[mobyIsMe]

Hi, Carlos Miguel Correia da Costa: After reading the conversation between you two， so all I have to do is change the database ：images of the currency I need complied to the according formats？ Is this right？ I don‘t understand What the metadata means， please help Me，Thanks.

[carlosmccosta]

Good morning,

The database contains the images of the banknotes in 3 different resolutions (currencyDB_mediumResolution, currencyDB_lowResolution, currencyDB_veryLowResolution). The 3 folders are stored here: https://github.com/carlosmccosta/Currency-Recognition/tree/master/CurrencyRecognition/imgs

The metadata is information attached to the database images, such as:

1 - The bitmasks for selecting the areas in which the keypoints will be searched in each database image (present in the 3 folders that I mentioned earlier, in which every image [name].png has a mask with filename [name]_mask.png - each pair of image and mask must have the same image resolution)

2 - The file which specifies the banknotes images to load (from the 3 folders I mentioned earlier) for building the recognition database, along with their currency value: https://github.com/carlosmccosta/Currency-Recognition/blob/master/CurrencyRecognition/imgs/listCurrencyDB.txt Each line provides information about: image filename | currency value | color for drawing banknote border in RGB

3 - The file that specifies the test images that will be used to evaluate the recognition system: https://github.com/carlosmccosta/Currency-Recognition/blob/master/CurrencyRecognition/imgs/listTestDB.txt which will be loaded from this folder: https://github.com/carlosmccosta/Currency-Recognition/tree/master/CurrencyRecognition/imgs/testDB

4 - An image with acceptable banknote shapes in different perspectives (for computing the contour aspect ratio and circularity for filtering the recognition results): https://github.com/carlosmccosta/Currency-Recognition/blob/master/CurrencyRecognition/imgs/currencyDBShapes.png

For more info please read: https://www.researchgate.net/publication/301888713_Recognition-of-Banknotes-in-Multiple-Perspectives-Using-Selective-Feature-Matching-and-Shape-Analysis-Presentation https://www.researchgate.net/publication/301888929_Recognition_of_Banknotes_in_Multiple_Perspectives_Using_Selective_Feature_Matching_and_Shape_Analysis

Have a nice day :)

[mobyIsMe]

Thanks. Do u think this could be integrated to a model and converted into the coreML SDK of iOS 11 developed by Apple? I am trying doing this king of thing: a currency recognition app in iOS with coreML SDK.

Have a nice day :)

[carlosmccosta]

Good afternoon,

Since OpenCV is supported in iOS it should be straightforward to integrate the approach I developed.

The core of my recognition system relies on the findHomography function of OpenCV: http://docs.opencv.org/master/d9/d0c/group__calib3d.html#ga4abc2ece9fab9398f2e560d53c8c9780

It seems coreML also has similar functionality available: https://developer.apple.com/documentation/vision/vnhomographicimageregistrationrequest

You could start by testing the vision modules of coreML and see if they are enough for what you are trying to achieve: https://developer.apple.com/documentation/vision

Have a nice day :)

[mobyIsMe]

Thanks very much，I will try it.

[mobyIsMe]

Hi， I have tried the vision api u suggested before， I input two images which are very similar to each other, and I got this matrix like this: (matrix_float3x3) matrix = { columns = { [0] = (0.843727767, -0.130427986, -0.000717610063, 0) [1] = (-0.00714870868, 0.965400338, 0.000267642288, 0) [2] = (5.10606575, -0.413659096, 0.934999823, 0.00000000000000000000000000000000000000000000140129846) } }

Besides, I input two images which are not similar, I got this result： (matrix_float3x3) matrix = { columns = { [0] = (0.843611419, 0.11978922, 0.000843094138, 0) [1] = (0.131927058, 1.27364492, 0.00245486898, 0) [2] = (-17.330986, -63.8090439, 0.454459071, 0.00000000000000000000000000000000000000000000140129846) }

Could u give me some tips about this results? Here is the code about how I use the vision API:

` -( matrix_float3x3)predictWithVisionFromImage:(UIImage )imageTarget toReferenceImage:(UIImage)imageRefer{

UIImage *scaledImageTarget = [imageTarget scaleToSize:CGSizeMake(224, 224)];
CVPixelBufferRef bufferTarget = [imageTarget pixelBufferFromCGImage:scaledImageTarget];
UIImage *scaledImageRefer = [imageRefer scaleToSize:CGSizeMake(224, 224)];
CVPixelBufferRef bufferRefer = [imageRefer pixelBufferFromCGImage:scaledImageRefer];

VNHomographicImageRegistrationRequest* request = [[VNHomographicImageRegistrationRequest alloc]initWithTargetedCVPixelBuffer:bufferTarget completionHandler:nil];

VNHomographicImageRegistrationRequest* imageRequest = (VNHomographicImageRegistrationRequest*)request;
VNImageRequestHandler* handler = [[VNImageRequestHandler alloc]initWithCVPixelBuffer:bufferRefer options:@{}];
[handler performRequests:@[imageRequest] error:nil];
NSArray* resultsArr = imageRequest.results;
VNImageHomographicAlignmentObservation* firstObservation = [resultsArr firstObject];
return firstObservation.warpTransform;

} ` there are two images: target image and reference image, this function is designed to calculate the similarity between they two, and result is displayed in the form of matrix.

[carlosmccosta]

Good morning,

The matrix returned is an homography that maps pixels in one image to the other. It can be used to compute the contour of your banknote. For example, if the reference image is a banknote occupying all your image in a front view, then you can use the homography to perform a perspective transform and find where the 4 reference image corners are in the other image (which could have been captured with your smartphone camera, for example).

Examples: http://docs.opencv.org/master/d7/dff/tutorial_feature_homography.html https://github.com/carlosmccosta/Currency-Recognition/blob/master/CurrencyRecognition/src/ImageAnalysis/DetectorResult.cpp#L28

More info: https://ags.cs.uni-kl.de/fileadmin/inf_ags/3dcv-ws11-12/3DCV_WS11-12_lec04.pdf https://en.wikipedia.org/wiki/Homography_(computer_vision)

Have a nice day :)

[mobyIsMe]

Thanks a lot. I am also trying the deep learning algorithm doing this. What is the recognition accuracy with your methods?

[carlosmccosta]

Good morning,

In the testing database with 80 images the system managed to correctly identify all of them. If a unique section of a banknote is visible, it is very likely that it will be recognized correctly.

Have a nice day :)