gray and silver car not working.

[moxgeek]

hello, i really like your work and your approach to resolving the challenges in this field. however, while using your solution, I found several issues while, gray, and silver-painted car will not work, or may work but will be detected several plate even if it's not a plate. i can share with you the pictures and videos i have been using if you wanna improve your solution . thank you

[apoorva-dave]

Yes in this case you might have to adjust plate dimensions so that just 1 license plate is detected. You can try editing in DetectPlate.py. Sure you can share the images I will try and have a look.

[moxgeek]

thank you alot . if you have an email where I can send you the image data it will be good (some of the pictures are confidential ).

also what if the picture was really near to the plate. for example :

do i need to change the following values :

plate_dimensions = (0.03*label_image.shape[0], 0.08*label_image.shape[0], 0.15*label_image.shape[1], 0.3*label_image.shape[1])
plate_dimensions2 = (0.08*label_image.shape[0], 0.2*label_image.shape[0], 0.15*label_image.shape[1], 0.4*label_image.shape[1])
min_height, max_height, min_width, max_width = plate_dimensions

cause i tried to play with those values but at the end didn't work.

[moxgeek]

i did change the values to :

plate_dimensions = (0.15*label_image.shape[0], 0.9*label_image.shape[0], 0.6*label_image.shape[1], 0.9*label_image.shape[1])
plate_dimensions2 = (0.15*label_image.shape[0], 0.9*label_image.shape[0], 0.6*label_image.shape[1], 0.9*label_image.shape[1])

and it's working fine.

however i right now i use your code just to crop the plate. detecting white, silver, and gray car will not work, only if we change the grayscal.

[apoorva-dave]

I could not understand. So you mean that you got it working by changing the dimensions, but it is not working for gray car image?

[moxgeek]

exactly. the two demo videos you used in your sample was a red and black car , however using gray. silver or white car image or video will not show a suitable result. i thing because the gray scale in this case will not be as we wanted ( the first display shows me a 80% white image ) so i assume the algorithm start to look for plate and detect some that is not really a plate . try to use the image of plate that i attached before ,you can see that this image will detect a plate but it's not correct. also i can send you car images that have the issues, but as a said before i can't post them here.

[apoorva-dave]

ohhh this is interesting. I will check with the image that you have attached, in case I need more images I will let you know

[moxgeek]

this is an other example about the silver problem , the plate will dispere once applying as_gray=true

[KunikaValecha]

Hi @moxgeek , I guess the problem is due to binarization of graysacle image. In this code the method of binarization used is otsu. Otsu finds a suitable mean value to binarize the pixels in image. For example, in a grayscale image pixel intensity is highest at 0 and 255, then otsu will say all values before 127 will be black and all values after 127 will be white, this value 127 is called retVal, so in simple thresholding we decide the retval whereas in otsu, retval is decided by machine seeing the histogram of pixel intensity. Now coming to this problem some of the colours of car like gray , darkgray, lightgray, silver have respective values as 128, 169, 211, 192 which are towards peak of white so when otsu will binarize these colours it will put it in category of white and hence the number plate will get mix with the car colour.

Coming to the solution for gray, silver and other colours u can solve the problem by keeping the retval as 249. But for white the problem may persist. Also, if by chance, the number plate looks a little darker in scene due to shadow then it will again be not detected as it wont be 255 on grayscale.

So, universal solution is either to use adaptive thresholding, which is a function of cv2 as well as skimage just like otsu, or i suggest u to first perform car detection followed by number plate detection with some suitable CNN and then perform component analysis once number plate is getting detected.

[KunikaValecha]

@moxgeek , Also you can play around with hough transform and image preprocessing (including noise removal, image whitening, sobbel filter, etc) techniques if you aint willing to dive into CNNs for the purpose.

[moxgeek]

or i suggest u to first perform car detection followed by number plate detection with some suitable CNN and then perform component analysis once number plate is getting detected.

that exactly what i did , i used yolo to detect the car ( with classification if it's a car or truck , moto ....) , and after i detect the plate. i send it to DetectPlate.py ( i wrappe it into django as a REST API ) , and i changes the height and weight and it's work up to 90%

( i know this issues #19) is nothing to do with the list of issues that im listing but just to clarify some error that can happen ) now im facing an issues with the Segmentation, specifically with letters composed by two parts , suck as 'i' or some arabic letters that have dots ض ش ق ف غ ظ ن ؤ ئ أ إ ة what i did to resolve this last issues is to take the same height of first char and apply it to the other , but with that way it reduce the learning detection cause in this case we will find some area in the char that are blank and the model was't trainined for that. also if the char was the first it will not work.

there is an other issues that i did resolve , some plates are regioned ( the spliter | are big enough that the DetectPlate.py script can see a plate but it's only i part of .) , i did resolve it by using a dictionary for all regions and merge them to only one . ( i can share my solution with you , it will improve the detection and will not effect the current solution )

[KunikaValecha]

or i suggest u to first perform car detection followed by number plate detection with some suitable CNN and then perform component analysis once number plate is getting detected.

that exactly what i did , i used yolo to detect the car ( with classification if it's a car or truck , moto ....) , and after i detect the plate. i send it to DetectPlate.py ( i wrappe it into django as a REST API ) , and i changes the height and weight and it's work up to 90%

( i know this issues #19) is nothing to do with the list of issues that im listing but just to clarify some error that can happen ) now im facing an issues with the Segmentation, specifically with letters composed by two parts , suck as 'i' or some arabic letters that have dots ض ش ق ف غ ظ ن ؤ ئ أ إ ة what i did to resolve this last issues is to take the same height of first char and apply it to the other , but with that way it reduce the learning detection cause in this case we will find some area in the char that are blank and the model was't trainined for that. also if the char was the first it will not work.

there is an other issues that i did resolve , some plates are regioned ( the spliter | are big enough that the DetectPlate.py script can see a plate but it's only i part of .) , i did resolve it by using a dictionary for all regions and merge them to only one . ( i can share my solution with you , it will improve the detection and will not effect the current solution )

For the text recognition part you may opt to go with OCR. That will make your work easy but I doubt it will increase the processing time as you will be using two complex Neural networks, one in YOLO and another in OCR. If you are fine with increase in time of computing, then you must checkout OCR once.

And, you can share your code with me on github itself using a public or private repo whatever suits you, my handle is @Kunika21 .

[moxgeek]

exactly, the problem using OCR gonna make this part worst in term of computation and realtime detection. so i did it changing some lines in the segmentation char and it's work fine ( the hack that i did is related to the hight of those chars ,if there are less then avg by 20% , i add some extra +15 on y0 value.

i think that the result i have for the moment is great, im working on the first part ( detection the car with darknet ) espacially that darknet wights is not supported for android , even with tenserflow demo. i tried darkflow but the model that im using for the car and plate detection is created with tiny yolo cfg, and that not supported in darflow, and even for some other project like DW2TF, i think that im doing something wrong . espacily that the car and plate detection is slow (1fram/s).

anyways ,this project is really good and simple to understand and refactoring ,however improvement by @apoorva-dave can be suitable :)

[apoorva-dave]

@moxgeek @Kunika21 hey apologies for not replying. I have been quite busy and could not get time to go through the issue. @moxgeek I hope your problem was solved by @Kunika21. Feel free to contribute to the project and raise a pull request.