search

ria-com / nomeroff-net

Nomeroff Net. Automatic numberplate recognition system.
GNU General Public License v3.0
459 stars 159 forks source link

Selecting the wrong region #214

Closed TryAnotherName closed 2 years ago

TryAnotherName commented 2 years ago

Hello. I really liked your library. I'm not very experienced and I don't understand a lot of things in the code. I ran into a problem that many Russian numbers are erroneously identified as Soviet Union numbers. Could this affect the quality? Is it possible to disable the region detection and use only the Russian model? 1

dimabendera commented 2 years ago

Added example how to disable the region detection and use only the Russian model through pipeline https://github.com/ria-com/nomeroff-net/blob/master/examples/ju/inference/get-started-demo-ocr-custom.ipynb

TryAnotherName commented 2 years ago

Added example how to disable the region detection and use only the Russian model through pipeline https://github.com/ria-com/nomeroff-net/blob/master/examples/ju/inference/get-started-demo-ocr-custom.ipynb

Wow, cool. Thanks a lot!

TryAnotherName commented 2 years ago

I tried including your example in my code, but I got an error. When using the photo "test_2.jpeg" of about 720p quality, an error appeared, as in the first photo. When using a photo quality of approximately 1080p, the number was determined as Soviet, as in the second photo. But if you use a 2k quality photo, the number was already determined as Russian. The text of the number was determined incorrectly in all cases. Can this be somehow corrected by the settings or is it just because the training is done only on a good quality photo or the photo itself is initially of poor quality?

p.s. We can switch to Russian, if it's more convenient for you.

Screenshot1 problem_1 Screenshot 2 problem_2

dimabendera commented 2 years ago

Can you send the original photo for testing?

TryAnotherName commented 2 years ago

Yes, of course 2

TryAnotherName commented 2 years ago

Can you send the original photo for testing?

Hello. Did anything work?

dimabendera commented 2 years ago

In the new release, we updated the models, now ru ocr works better (now your example is working correctly)

TryAnotherName commented 2 years ago

In the new release, we updated the models, now ru ocr works better (now your example is working correctly)

Hello again. I'm back to you) I collected a selection of photos of different quality with numbers from the Internet. There are a couple of new bugs. I found out that the errors are associated with certain photos, but have not yet determined which ones. I send you a link to the sample and screenshots of errors. I also have other questions like this. How to use my trained model? In your example, I tried to insert the path to the model trained for the previous version of the library, but I got an error, as in photo №3. And a question about the automatic marking of the number and the creation of a sample. I saw a closed thread with a similar question, but the answer in it referred to a previous version of the library. As I understand it, you did not add this utility in the new version? Is it still relevant or is it worth waiting for the update and its appearance for the updated version of the library?

https://drive.google.com/drive/folders/1M0lMYeVRhLpxuujN5882qfnzAK-bVYHk?usp=sharing

Error 1 image Error 2 image Error 3 image

p. s. Are you tired of me yet? :)

dimabendera commented 2 years ago

And a question about the automatic marking of the number and the creation of a sample. I saw a closed thread with a similar question, but the answer in it referred to a previous version of the library.

Can you post a thread with a similar question to understand what we are talking about?

As for the third error, unfortunately the ocr models from previous versions are not compatible with the current one.

dimabendera commented 2 years ago

p. s. Are you tired of me yet? :)

Thanks for helping us make our product better!

TryAnotherName commented 2 years ago

Can you post a thread with a similar question to understand what we are talking about?

https://github.com/ria-com/nomeroff-net/issues/170

dimabendera commented 2 years ago

Can you post a thread with a similar question to understand what we are talking about?

170

Usage example: https://github.com/ria-com/nomeroff-net/blob/master/examples/ju/dataset_tools/auto_number_grab.ipynb

auto_number_grab tool source code https://github.com/ria-com/nomeroff-net/blob/master/nomeroff_net/tools/datasets_tools.py#L118

TryAnotherName commented 2 years ago

As for the third error, unfortunately the ocr models from previous versions are not compatible with the current one.

That is, in order to use my own model, did I correctly indicate everything in the screenshot with error No. 3?

Usage example: https://github.com/ria-com/nomeroff-net/blob/master/examples/ju/dataset_tools/auto_number_grab.ipynb

auto_number_grab tool source code https://github.com/ria-com/nomeroff-net/blob/master/nomeroff_net/tools/datasets_tools.py#L118

Thanks. Sorry I didn't notice. Do I understand correctly that for the correct work, it is enough for me to indicate the correct number in the photo title?

TryAnotherName commented 2 years ago

Hello. I tried to train my model and got this error. Interestingly, the letters in the class ru(OCR) do not match what is written in the execution log.

PATH_TO_DATASET = os.path.abspath("/content/autoriaNumberplateOcrRu-2021-09-01")

image

dimabendera commented 2 years ago

Thanks. Sorry I didn't notice. Do I understand correctly that for the correct work, it is enough for me to indicate the correct number in the photo title?

No, the correct license plate is taken from photos_dataset.csv Example https://github.com/ria-com/nomeroff-net/blob/master/data/dataset/Detector/autoria_numberplate_dataset_example/train/photos_dataset.csv

But you can change the script to your liking.

dimabendera commented 2 years ago

image

Can you show the entire error stack?

Try just Install the latest version of pillow pip3 install --upgrade Pillow. And Restart your runtime

TryAnotherName commented 2 years ago

Thanks. Sorry I didn't notice. Do I understand correctly that for the correct work, it is enough for me to indicate the correct number in the photo title?

No, the correct license plate is taken from photos_dataset.csv Example https://github.com/ria-com/nomeroff-net/blob/master/data/dataset/Detector/autoria_numberplate_dataset_example/train/photos_dataset.csv

But you can change the script to your liking.

Thank you, I already found and figured out how to do this conveniently.

TryAnotherName commented 2 years ago

Can you show the entire error stack?

Try just Install the latest version of pillow pip3 install --upgrade Pillow. And Restart your runtime

It seems that the last error is related to the update of some library. Same error in auto_grabber. I work in google.collab and every time I start I just setup the requirements.txt. image

TryAnotherName commented 2 years ago

This is how the error looks in the train module.

image

image

dimabendera commented 2 years ago

Can you send the result of the commands?

python --version
pip --version
pip list

I will try to reproduce the error

TryAnotherName commented 2 years ago

Can you send the result of the commands?

Python 3.7.13
pip 21.1.3 from /usr/local/lib/python3.7/dist-packages/pip (python 3.7)
Package                       Version
----------------------------- ---------------------
absl-py                       1.0.0
aiohttp                       3.8.1
aiosignal                     1.2.0
alabaster                     0.7.12
albumentations                0.1.12
altair                        4.2.0
appdirs                       1.4.4
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
arviz                         0.11.4
astor                         0.8.1
astropy                       4.3.1
astunparse                    1.6.3
async-timeout                 4.0.2
asyncio                       3.4.3
asynctest                     0.13.0
atari-py                      0.2.9
atomicwrites                  1.4.0
attrs                         21.4.0
audioread                     2.1.9
autograd                      1.3
Babel                         2.9.1
backcall                      0.2.0
beautifulsoup4                4.6.3
bleach                        4.1.0
blis                          0.4.1
bokeh                         2.3.3
Bottleneck                    1.3.4
branca                        0.4.2
bs4                           0.0.1
CacheControl                  0.12.10
cached-property               1.5.2
cachetools                    4.2.4
catalogue                     1.0.0
certifi                       2021.10.8
cffi                          1.15.0
cftime                        1.6.0
chardet                       3.0.4
charset-normalizer            2.0.12
click                         7.1.2
cloudpickle                   1.3.0
cmake                         3.12.0
cmdstanpy                     0.9.5
colorcet                      3.0.0
colorlover                    0.3.0
community                     1.0.0b1
contextlib2                   0.5.5
convertdate                   2.4.0
coverage                      3.7.1
coveralls                     0.5
crcmod                        1.7
cufflinks                     0.17.3
cupy-cuda111                  9.4.0
cvxopt                        1.2.7
cvxpy                         1.0.31
cycler                        0.11.0
cymem                         2.0.6
Cython                        0.29.28
daft                          0.0.4
dask                          2.12.0
datascience                   0.10.6
debugpy                       1.0.0
decorator                     4.4.2
defusedxml                    0.7.1
descartes                     1.1.0
dill                          0.3.4
distributed                   1.25.3
dlib                          19.18.0
dm-tree                       0.1.6
docopt                        0.6.2
docutils                      0.17.1
dopamine-rl                   1.0.5
earthengine-api               0.1.303
easydict                      1.9
ecos                          2.0.10
editdistance                  0.5.3
en-core-web-sm                2.2.5
entrypoints                   0.4
ephem                         4.1.3
et-xmlfile                    1.1.0
fa2                           0.3.5
fastai                        1.0.61
fastdtw                       0.3.4
fastprogress                  1.0.2
fastrlock                     0.8
fbprophet                     0.7.1
feather-format                0.4.1
filelock                      3.6.0
firebase-admin                4.4.0
fix-yahoo-finance             0.0.22
Flask                         1.1.4
flatbuffers                   2.0
folium                        0.8.3
frozenlist                    1.3.0
fsspec                        2022.2.0
future                        0.18.2
gast                          0.5.3
GDAL                          2.2.2
gdown                         4.2.2
gensim                        3.6.0
geographiclib                 1.52
geopy                         1.17.0
gevent                        21.12.0
gin-config                    0.5.0
gitdb                         4.0.9
GitPython                     3.1.18
glob2                         0.7
google                        2.0.3
google-api-core               1.26.3
google-api-python-client      1.12.11
google-auth                   1.35.0
google-auth-httplib2          0.0.4
google-auth-oauthlib          0.4.6
google-cloud-bigquery         1.21.0
google-cloud-bigquery-storage 1.1.0
google-cloud-core             1.0.3
google-cloud-datastore        1.8.0
google-cloud-firestore        1.7.0
google-cloud-language         1.2.0
google-cloud-storage          1.18.1
google-cloud-translate        1.5.0
google-colab                  1.0.0
google-pasta                  0.2.0
google-resumable-media        0.4.1
googleapis-common-protos      1.56.0
googledrivedownloader         0.4
graphviz                      0.10.1
greenlet                      1.1.2
grpcio                        1.44.0
gspread                       3.4.2
gspread-dataframe             3.0.8
gym                           0.17.3
h5py                          3.1.0
HeapDict                      1.0.1
hijri-converter               2.2.3
holidays                      0.10.5.2
holoviews                     1.14.8
html5lib                      1.0.1
httpimport                    0.5.18
httplib2                      0.17.4
httplib2shim                  0.0.3
humanize                      0.5.1
hyperopt                      0.1.2
ideep4py                      2.0.0.post3
idna                          2.10
imageio                       2.4.1
imagesize                     1.3.0
imbalanced-learn              0.8.1
imblearn                      0.0
imgaug                        0.4.0
importlib-metadata            4.11.3
importlib-resources           5.4.0
imutils                       0.5.4
inflect                       2.1.0
iniconfig                     1.1.1
intel-openmp                  2022.0.2
intervaltree                  2.1.0
ipykernel                     4.10.1
ipython                       5.5.0
ipython-genutils              0.2.0
ipython-sql                   0.3.9
ipywidgets                    7.7.0
itsdangerous                  1.1.0
jax                           0.3.4
jaxlib                        0.3.2+cuda11.cudnn805
jedi                          0.18.1
jieba                         0.42.1
Jinja2                        2.11.3
joblib                        1.1.0
jpeg4py                       0.1.4
jsonpickle                    2.1.0
jsonschema                    4.3.3
jupyter                       1.0.0
jupyter-client                5.3.5
jupyter-console               5.2.0
jupyter-core                  4.9.2
jupyterlab-pygments           0.1.2
jupyterlab-widgets            1.1.0
kaggle                        1.5.12
kapre                         0.3.7
keras                         2.8.0
Keras-Preprocessing           1.1.2
keras-vis                     0.4.1
kiwisolver                    1.4.0
korean-lunar-calendar         0.2.1
libclang                      13.0.0
librosa                       0.8.1
lightgbm                      2.2.3
llvmlite                      0.34.0
lmdb                          0.99
LunarCalendar                 0.0.9
lxml                          4.2.6
Markdown                      3.3.6
MarkupSafe                    2.0.1
matplotlib                    3.2.2
matplotlib-inline             0.1.3
matplotlib-venn               0.11.6
missingno                     0.5.1
mistune                       0.8.4
mizani                        0.6.0
mkl                           2019.0
mlxtend                       0.14.0
modelhub-client               1.1.0
more-itertools                8.12.0
moviepy                       0.2.3.5
mpmath                        1.2.1
msgpack                       1.0.3
multidict                     6.0.2
multiprocess                  0.70.12.2
multitasking                  0.0.10
murmurhash                    1.0.6
music21                       5.5.0
natsort                       5.5.0
nbclient                      0.5.13
nbconvert                     5.6.1
nbformat                      5.2.0
nest-asyncio                  1.5.4
netCDF4                       1.5.8
networkx                      2.6.3
nibabel                       3.0.2
nltk                          3.2.5
notebook                      5.3.1
numba                         0.51.2
numexpr                       2.8.1
numpy                         1.21.5
nvidia-ml-py3                 7.352.0
oauth2client                  4.1.3
oauthlib                      3.2.0
okgrade                       0.4.3
opencv-contrib-python         4.1.2.30
opencv-python                 4.1.2.30
openpyxl                      3.0.9
opt-einsum                    3.3.0
osqp                          0.6.2.post0
packaging                     21.3
palettable                    3.3.0
pandas                        1.3.5
pandas-datareader             0.9.0
pandas-gbq                    0.13.3
pandas-profiling              1.4.1
pandocfilters                 1.5.0
panel                         0.12.1
param                         1.12.0
parso                         0.8.3
pathlib                       1.0.1
patsy                         0.5.2
pep517                        0.12.0
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        9.0.1
pip                           21.1.3
pip-tools                     6.2.0
plac                          1.1.3
plotly                        5.5.0
plotnine                      0.6.0
pluggy                        0.7.1
pooch                         1.6.0
portpicker                    1.3.9
prefetch-generator            1.0.1
preshed                       3.0.6
prettytable                   3.2.0
progressbar2                  3.38.0
prometheus-client             0.13.1
promise                       2.3
prompt-toolkit                1.0.18
protobuf                      3.17.3
psutil                        5.4.8
psycopg2                      2.7.6.1
ptyprocess                    0.7.0
py                            1.11.0
pyarrow                       6.0.1
pyasn1                        0.4.8
pyasn1-modules                0.2.8
pycocotools                   2.0.4
pycparser                     2.21
pyct                          0.4.8
pydata-google-auth            1.4.0
pyDeprecate                   0.3.1
pydot                         1.3.0
pydot-ng                      2.0.0
pydotplus                     2.0.2
PyDrive                       1.3.1
pyemd                         0.5.1
pyerfa                        2.0.0.1
pyglet                        1.5.0
Pygments                      2.6.1
pygobject                     3.26.1
pymc3                         3.11.4
PyMeeus                       0.5.11
pymongo                       4.0.2
pymystem3                     0.2.0
PyOpenGL                      3.1.6
pyparsing                     3.0.7
pyrsistent                    0.18.1
pysndfile                     1.3.8
PySocks                       1.7.1
pystan                        2.19.1.1
pytest                        3.6.4
python-apt                    0.0.0
python-chess                  0.23.11
python-dateutil               2.8.2
python-louvain                0.16
python-slugify                6.1.1
python-utils                  3.1.0
pytorch-lightning             1.5.10
PyTurboJPEG                   1.6.6
pytz                          2018.9
pyviz-comms                   2.1.0
PyWavelets                    1.3.0
PyYAML                        6.0
pyzmq                         22.3.0
qdldl                         0.1.5.post0
qtconsole                     5.2.2
QtPy                          2.0.1
regex                         2019.12.20
requests                      2.23.0
requests-oauthlib             1.3.1
resampy                       0.2.2
rpy2                          3.4.5
rsa                           4.8
scikit-image                  0.18.3
scikit-learn                  1.0.2
scipy                         1.4.1
screen-resolution-extra       0.0.0
scs                           3.2.0
seaborn                       0.11.2
semver                        2.13.0
Send2Trash                    1.8.0
setuptools                    59.5.0
setuptools-git                1.2
Shapely                       1.8.1.post1
simplegeneric                 0.8.1
six                           1.15.0
sklearn                       0.0
sklearn-pandas                1.8.0
smart-open                    5.2.1
smmap                         5.0.0
snowballstemmer               2.2.0
sortedcontainers              2.4.0
SoundFile                     0.10.3.post1
soupsieve                     2.3.1
spacy                         2.2.4
Sphinx                        1.8.6
sphinxcontrib-serializinghtml 1.1.5
sphinxcontrib-websupport      1.2.4
SQLAlchemy                    1.4.32
sqlparse                      0.4.2
srsly                         1.0.5
statsmodels                   0.10.2
sympy                         1.7.1
tables                        3.7.0
tabulate                      0.8.9
tblib                         1.7.0
tenacity                      8.0.1
tensorboard                   2.8.0
tensorboard-data-server       0.6.1
tensorboard-plugin-wit        1.8.1
tensorflow                    2.8.0
tensorflow-datasets           4.0.1
tensorflow-estimator          2.8.0
tensorflow-gcs-config         2.8.0
tensorflow-hub                0.12.0
tensorflow-io-gcs-filesystem  0.24.0
tensorflow-metadata           1.7.0
tensorflow-probability        0.16.0
termcolor                     1.1.0
terminado                     0.13.3
testpath                      0.6.0
text-unidecode                1.3
textblob                      0.15.3
Theano-PyMC                   1.1.2
thinc                         7.4.0
threadpoolctl                 3.1.0
tifffile                      2021.11.2
tomli                         2.0.1
toolz                         0.11.2
torch                         1.10.0+cu111
torchaudio                    0.10.0+cu111
torchmetrics                  0.7.3
torchsummary                  1.5.1
torchtext                     0.11.0
torchvision                   0.11.1+cu111
tornado                       5.1.1
tqdm                          4.63.0
traitlets                     5.1.1
tweepy                        3.10.0
typeguard                     2.7.1
typing-extensions             3.10.0.2
tzlocal                       1.5.1
ujson                         5.1.0
uritemplate                   3.0.1
urllib3                       1.24.3
vega-datasets                 0.9.0
wasabi                        0.9.0
wcwidth                       0.2.5
webencodings                  0.5.1
Werkzeug                      1.0.1
wheel                         0.37.1
widgetsnbextension            3.6.0
wordcloud                     1.5.0
wrapt                         1.14.0
xarray                        0.18.2
xgboost                       0.90
xkit                          0.0.0
xlrd                          1.1.0
xlwt                          1.3.0
yarl                          1.7.2
yellowbrick                   1.4
zict                          2.1.0
zipp                          3.7.0
zope.event                    4.5.0
zope.interface                5.4.0
TryAnotherName commented 2 years ago

No, the correct license plate is taken from photos_dataset.csv Example https://github.com/ria-com/nomeroff-net/blob/master/data/dataset/Detector/autoria_numberplate_dataset_example/train/photos_dataset.csv

But you can change the script to your liking.

Can you please explain what the auto_number_grab does? It seemed to me that it was enough for him to give a photo and a label of the correct number, and the model herself would find the number in the photo, cut it out and save it in a separate folder, creating an annotation file. I didn't create a .json file for each photo like in the train folder and I don't know how. But I checked with the files from the example and the error persisted.

dimabendera commented 2 years ago

As a result, you will get cut and aligned license plates in the res_dir folder in the format {origbasename}{numberplate}.png

Next, you must create image annotation json files and split the dataset into train, test, and val.

For example, you can do this with our node-js scripts, correcting them: https://github.com/ria-com/nomeroff-net/blob/master/moderation/app_console/controllers/defaultController.js#L31 https://github.com/ria-com/nomeroff-net/blob/master/moderation/app_console/controllers/defaultController.js#L114 Or write your own.

An example of the final dataset https://github.com/ria-com/nomeroff-net/tree/master/data/dataset/TextDetector/ocr_example

dimabendera commented 2 years ago

Sorry, we can't reproduce your bug with Pillow. But we will be glad if you share, having found a solution, as it may be useful to someone else.

Try looking at similar issues https://github.com/facebookresearch/detectron2/issues/2231

TryAnotherName commented 2 years ago

As a result, you will get cut and aligned license plates in the res_dir folder in the format {origbasename}{numberplate}.png

Next, you must create image annotation json files and split the dataset into train, test, and val.

For example, you can do this with our node-js scripts, correcting them: https://github.com/ria-com/nomeroff-net/blob/master/moderation/app_console/controllers/defaultController.js#L31 https://github.com/ria-com/nomeroff-net/blob/master/moderation/app_console/controllers/defaultController.js#L114 Or write your own.

An example of the final dataset https://github.com/ria-com/nomeroff-net/tree/master/data/dataset/TextDetector/ocr_example

I meant a little different. The Detector\autoria_numberplate_dataset_example\train folder contains the via_region_data.json file. Does it need to be created using VIA, or is it only needed for manual markup? Because it seemed to me that the model itself should find the number, and it was enough for me to write the correct number label. But thanks for explaining how to create an annotation file for each photo, since I thought they should be created automatically too.

TryAnotherName commented 2 years ago

Sorry, we can't reproduce your bug with Pillow. But we will be glad if you share, having found a solution, as it may be useful to someone else.

Try looking at similar issues facebookresearch/detectron2#2231

Hmm. I'm a fool. Sorry for wasting your time. I thought that in the collab, restarting the environment resets all local files and all installed libraries. After installing the requirements.txt, you just had to click on the restart of the environment and nothing was reset. But now everything is working. Sorry again.

TryAnotherName commented 2 years ago

Hello. There was an error when running auto_number_grab. I can guess why this might be. Most likely, the model could not find the number in the photo. Could you add an exclusion of such photos and display a title or some kind of label so that this photo can be marked up manually?

image

TryAnotherName commented 2 years ago

Hello. There was an error when running auto_number_grab. I can guess why this might be. Most likely, the model could not find the number in the photo. Could you add an exclusion of such photos and display a title or some kind of label so that this photo can be marked up manually?

image

Hello. Haven't looked at anything about the error in the last comment?

TryAnotherName commented 2 years ago

Added example how to disable the region detection and use only the Russian model through pipeline https://github.com/ria-com/nomeroff-net/blob/master/examples/ju/inference/get-started-demo-ocr-custom.ipynb

There is another question. What do I need to write to use my model to define the license plate frame in the pipeline? Could you add such an example with both a specific region and a specific model selection?

TryAnotherName commented 2 years ago

Hello. Did you find anything?