get the mAP of hezarai/crnn-fa-64x256-license-plate-recognition

[davoodap]

Hello. First of all, I wanted to thank you for your very good and practical project. I wanted to know how to get the mAP of model(hezarai/crnn-fa-64x256-license-plate-recognition) on my dataset. Thank you for helping .

[arxyzan]

Hi @davoodap thank you for your nice feedback! Actually the mAP metric is for object detectino models that return a bounding box. The model you mentioned is for a step after that; meaning that you need a plate detectin model in advance to detect the plate's bounding box in an image and pass its output to the OCR model. Unfortunately, we do not currently provide a model for plate detection, but it's pretty easy to train one using YOLO.

[davoodap]

@arxyzan Thank you very much for your quick response. If I want to know the overall accuracy of the model on a dataset, what is the way?

[arxyzan]

@davoodap Thanks, Well, to get the accuracy of an OCR model you can use the CER metric. This metric is also available in Hezar. (don't forget to update hezar to the latest version: pip install -U hezar)

Consider you have a dataframe (loaded from a CSV for example) with path and text columns.

import pandas as pd
from hezar.models import Model
from hezar.metrics import build_metric

dataset = pd.read_csv("dataset.csv")
model = Model.load("hezarai/crnn-fa-printed-96-long")
metric = build_metric("cer")

def predict_fn(path):
    predicted_text = model.predict(path)[0]["text"]
    return predicted_text

dataset["prediction"] = dataset["path"].apply(predict_fn)

predictions = dataset["predictions"]
targets = dataset["text"]

cer_value = metric.compute(predictions, targets)
print(cer_value)  # A good CER is less than 0.05. the lower, the better.

Note: I didn't test this code so it might have minor issues but the general procedure is as above.