Problem with running license plate ocr model Locally, without any internet access.

[claymore07]

Hi, I am trying to run the License Plate OCR model locally without any internet access since there are some arguments to make this possible. Below is the code for loading the model:

model = Model.load("hezarai/crnn-fa-64x256-license-plate-recognition", load_locally = True)

Moreover, this is the directory structure for my project:

Now, when I am trying to run, I get this error, which seems to be caused by path generation to load the preprocessing configuration yaml file:

Traceback (most recent call last): File "C:\Users\Gease\OLicensePlate\main.py", line 8, in <module> from util import * File "C:\Users\Gease\OLicensePlate\util.py", line 9, in <module> model = Model.load("hezarai/crnn-fa-64x256-license-plate-recognition", load_locally = True) File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\hezar\models\model.py", line 171, in load preprocessor = Preprocessor.load(hub_or_local_path, force_return_dict=True, cache_dir=cache_dir) File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\hezar\preprocessors\preprocessor.py", line 98, in load config = OmegaConf.load(config_file) File "C:\ProgramData\anaconda3\envs\cv2\lib\site-packages\omegaconf\omegaconf.py", line 189, in load with io.open(os.path.abspath(file_), "r", encoding="utf-8") as f: FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Gease\\OLicensePlate\\hezarai\\crnn-fa-64x256-license-plate-recognition\\preprocessor\\hezarai\\crnn-fa-64x256-license-plate-recognition\\image_processor_config.yaml'

Please if it possible, help me out on this topic. Best Regards

[arxyzan]

Hi @claymore07, thanks for submitting an issue. The structure in your image seems to be correct (except for the extra image_processor_config.yaml outside the preprocessor subfolder). Can you try to save your model as below?

from hezar.models import Model

model = Model.load("hezarai/crnn-fa-64x256-license-plate-recognition")
model.save("crnn-alpr-fa")  # save model files to a local path

model = Model.load("crnn-alpr/fa")  # there is no need for specifying `load_locally` since local files are prior to Hub files.

[claymore07]

Thanks for your kind response and guide. With your help now the model and the directories are saved at the root of my project directory, however when trying to load the model, in the last line of your code, it throws an error like below:

Traceback (most recent call last):
  File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\huggingface_hub\utils\_errors.py", line 270, in hf_raise_for_status
    response.raise_for_status()
  File "C:\ProgramData\anaconda3\envs\cv2\lib\site-packages\requests\models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/crnn-alpr/fa/resolve/main/model_config.yaml

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\Gease\OLicensePlate\main.py", line 8, in <module>
    from util import *
  File "C:\Users\Gease\OLicensePlate\util.py", line 9, in <module>
    model = Model.load("crnn-alpr/fa", load_locally = True)
  File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\hezar\models\model.py", line 134, in load
    config = ModelConfig.load(hub_or_local_path=hub_or_local_path, filename=config_filename, cache_dir=cache_dir)
  File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\hezar\configs.py", line 171, in load
    config_path = hf_hub_download(
  File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\huggingface_hub\utils\_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\huggingface_hub\file_download.py", line 1374, in hf_hub_download
    raise head_call_error
  File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\huggingface_hub\file_download.py", line 1247, in hf_hub_download
    metadata = get_hf_file_metadata(
  File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\huggingface_hub\utils\_validators.py", line 118, in _inner_fn
    return fn(*args, **kwargs)
  File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\huggingface_hub\file_download.py", line 1624, in get_hf_file_metadata
    r = _request_wrapper(
  File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\huggingface_hub\file_download.py", line 402, in _request_wrapper
    response = _request_wrapper(
  File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\huggingface_hub\file_download.py", line 426, in _request_wrapper
    hf_raise_for_status(response)
  File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\huggingface_hub\utils\_errors.py", line 320, in hf_raise_for_status
    raise RepositoryNotFoundError(message, response) from e
huggingface_hub.utils._errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-65d0caa5-7e7339a65dd63bd27ad27992;19c8ede4-43d3-43cc-ac94-25bb793db642)

Repository Not Found for url: https://huggingface.co/crnn-alpr/fa/resolve/main/model_config.yaml.
Please make sure you specified the correct `repo_id` and `repo_type`.
If you are trying to access a private or gated repo, make sure you are authenticated.
Invalid username or password.

[claymore07]

Also, I have tried this command too, but got the same error:

model = Model.load("crnn-alpr/fa", load_locally = True)

[arxyzan]

@claymore07 Hmm, this seems like a bug. I'll let you know when I figure it out very soon!

[arxyzan]

@claymore07 What version of Hezar are you using?

[claymore07]

Please accept my sincere appreciation for your time and effort; I am well aware of your hectic schedule and value your consideration. I place the uttermost importance on promptly executing this project offline. It is version "0.34.0" that I am utilizing. Thank you for your assistance.

[claymore07]

I tested on the latest version 0.36.0 and got the same error.

[arxyzan]

@claymore07 Okay, now I noticed that I gave you the wrong code. The correct path is crnn-alpr-fa but I wrote it as crnn-alpr/fa. So the working code would be as below:

from hezar.models import Model

model = Model.load("hezarai/crnn-fa-64x256-license-plate-recognition")
model.save("crnn-alpr-fa")  # save model files to a local path

model = Model.load("crnn-alpr-fa")  # there is no need for specifying `load_locally` since local files are prior to Hub files.

[claymore07]

Hi Sorry for the delay, I've tested the last code that you provided and got this error:

Traceback (most recent call last):
  File "C:\Users\Gease\OLicensePlate\tests.py", line 266, in <module>
    model = Model.load("crnn-alpr-fa")
  File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\hezar\models\model.py", line 169, in load
    preprocessor = Preprocessor.load(hub_or_local_path, force_return_dict=True, cache_dir=cache_dir)
  File "C:\Users\Gease\AppData\Roaming\Python\Python39\site-packages\hezar\preprocessors\preprocessor.py", line 98, in load
    config = OmegaConf.load(config_file)
  File "C:\ProgramData\anaconda3\envs\cv2\lib\site-packages\omegaconf\omegaconf.py", line 189, in load
    with io.open(os.path.abspath(file_), "r", encoding="utf-8") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Gease\\OLicensePlate\\crnn-alpr-fa\\preprocessor\\crnn-alpr-fa\\image_processor_config.yaml'

It is looking for crnn-alpr-fa inside the "preprocessor" directory!

[arxyzan]

@claymore07 Seems pretty strange since this code works everywhere I tested. Are you using Windows? Cause I don't have access to a Windows system.

[claymore07]

@arxyzan Yeah, I am using Windows 11. I can't find the cause but in the preprocessor.py the path for loading the config file produces a nested directory!

[claymore07]

I traced down the path generation to preprocess.py line 84 which in return uses the list_repo_files method from hub_utils.py. I couldn't understand the logic but this is where everything goes wrong! when in line 98 of the preprocess.py, I used hard-coded relative, everything smoothly. I hope with this info you can resolve this problem. For now, I will use the hard-coded path, Which is completely wrong, but it works.

I'll wait for your solution. Thanks for the time and effort you've put into building this amazing library. Best Regards

[arxyzan]

@claymore07 Nice try! The reason behind listing repo files is that the Preprocessor can actually load multiple preprocessors and return a PreprocessorContainer instance if there are more than one preprocessor in the repo. But it's quite strange that it does such a behavior in local mode. Thanks for the info. I'm currently looking into it to solve it as soon as possible.

[arxyzan]

@claymore07 I just tested the same code on a windows 10 system and it worked. I looked into the code line by line and still didn't figure out what's the cause to this bug. Can you paste the code you changed in the source please?

[claymore07]

I only changed line 98 in the preprocess.py from: config = OmegaConf.load(config_file) to:

config = OmegaConf.load("crnn-alpr-fa\preprocessor\image_processor_config.yaml")

[arxyzan]

@claymore07 Sorry for the delay. Did you figure out the problem? If not, can you please print the preprocessor_files at line 84 at https://github.com/hezarai/hezar/blob/7751ea26d0d2c1f52ceec964dc5abc5e1a669b7d/hezar/preprocessors/preprocessor.py#L84

I think this method is somehow generating wrong paths to look for.

[Mostafa79modaqeq]

@arxyzan Hi, I also have the same problem to run this model in Windows locally. Has anything been done to solve this problem?

[arxyzan]

Hi @Mostafa79modaqeq , thanks for reporting the issue. Can you please share the exact code you're running and the full error message?

[Mostafa79modaqeq]

@arxyzan Of course from hezar.models import Model model = Model.load("hezarai/crnn-fa-64x256-license-plate-recognition") model.save("crnn-alpr-fa") model = Model.load("crnn-alpr-fa")

FileNotFoundError Traceback (most recent call last) Cell In[6], line 1 ----> 1 model = Model.load("crnn-alpr-fa")

File c:\Users\mostafi\Projects\HezarTest\myenv\Lib\site-packages\hezar\models\model.py:169, in Model.load(cls, hub_or_local_path, load_locally, load_preprocessor, model_filename, config_filename, save_path, cache_dir, **kwargs) 167 # Load the preprocessor(s) 168 if load_preprocessor: --> 169 preprocessor = Preprocessor.load(hub_or_local_path, force_return_dict=True, cache_dir=cache_dir) 170 model.preprocessor = preprocessor 171 return model

File c:\Users\mostafi\Projects\HezarTest\myenv\Lib\site-packages\hezar\preprocessors\preprocessor.py:98, in Preprocessor.load(cls, hub_or_local_path, subfolder, force_return_dict, cache_dir, **kwargs) 90 else: 91 config_file = hf_hub_download( 92 hub_or_local_path, 93 filename=f, (...) 96 cache_dir=cache_dir, 97 ) ---> 98 config = OmegaConf.load(config_file) 99 name = config.get("name", None) 100 if name:

File c:\Users\mostafi\Projects\HezarTest\myenv\Lib\site-packages\omegaconf\omegaconf.py:189, in OmegaConf.load(file_) 186 from ._utils import get_yamlloader 188 if isinstance(file, (str, pathlib.Path)): --> 189 with io.open(os.path.abspath(file_), "r", encoding="utf-8") as f: 190 obj = yaml.load(f, Loader=get_yamlloader()) 191 elif getattr(file, "read", None):

FileNotFoundError: [Errno 2] No such file or directory: 'c:\Users\mostafi\Projects\HezarTest\crnn-alpr-fa\preprocessor\crnn-alpr-fa\image_processor_config.yaml'

[Adversarian]

I may have found the culprit!

The issue stems from this line in the list_repo_files method: https://github.com/hezarai/hezar/blob/7751ea26d0d2c1f52ceec964dc5abc5e1a669b7d/hezar/utils/hub_utils.py#L123

Windows paths use backslashes (\) as opposed to the forward slash (/) in POSIX-based paths. This line is specifically attempting to replace part of a string that's supposed to include a forward slash which simply doesn't exist in Windows and so the returned list of directories is corrupted and there's a repeated {hub_or_local_path} in the middle of the string.

Thankfully, we can leverage pathlib or os.path to let Python solve this issue internally on a per-OS basis.

Per this post, something like this might work:

for x in f:
    path = pathlib.Path(r)
    index = path.parts.index(hub_or_local_path)
    new_path = pathlib.Path('').joinpath(*path.parts[:index], *path.parts[index+1:], x)
    files.append(str(new_path))

Moreover, this exact issue can happen again in: https://github.com/hezarai/hezar/blob/7751ea26d0d2c1f52ceec964dc5abc5e1a669b7d/hezar/utils/hub_utils.py#L128 and the fix would be similar.

If cross-OS compatibility is a concern I should strongly recommend using python's built-in path handlers (like pathlib and os).

[arxyzan]

Thanks @Adversarian , you are absolutely right. Idk why I wrote it in that way but anyways, I fixed it in f665f9e and will do some other tests to be sure it's working fine and is ready for release, but for now, you can clone the repo at main and test it out. (Didn't test it myself on Windows but according to the reasons @Adversarian mentioned I think the new changes would fix it)

[Adversarian]

Thanks for the quick fix @arxyzan!

I just tested this on a Windows machine and the fix seems to have resolved the issue.

[arxyzan]

@claymore07 @Mostafa79modaqeq The problem seems to be fixed on main. You can install Hezar from source on main and test it and let me know.

[arxyzan]

@claymore07 @Mostafa79modaqeq Guys, this issue is finally fixed in v0.36.1 . You can install the latest version of hezar through pip to test it out. Sorry it took a long time. Special thanks to @Adversarian for his genius troubleshooting.