search

armaanpriyadarshan / Training-a-Custom-TensorFlow-2.X-Object-Detector

Learn how to train a TensorFlow Custom Object Detector with TensorFlow-GPU
82 stars 47 forks source link

Issues with Numpy version #28

Closed kaiwalya4850 closed 2 years ago

kaiwalya4850 commented 2 years ago

So, all of it is working perfectly well for me, the training, the saving of the model, and even the testing of the saved model(using the file TF-image-od.py), but for some reason, I cannot get the evaluation to work. I tried numpy 1.17.3 as mentioned but no luck, did it work for any of you? Which version should I use?

This is the error I get with NumPy 1.17.3. Also, when I use NumPy 1.17.3 apart from this error, I cannot run the model on an image and test it, it'll throw the same error.

2022-05-11 23:51:27.525133: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found 2022-05-11 23:51:27.525327: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd Traceback (most recent call last): File "model_main_tf2.py", line 31, in import tensorflow.compat.v2 as tf File "C:\Users\Kaiwalya\Anaconda3\envs\py38\lib\site-packages\tensorflow__init.py", line 37, in from tensorflow.python.tools import module_util as _module_util File "C:\Users\Kaiwalya\Anaconda3\envs\py38\lib\site-packages\tensorflow\python__init__.py", line 37, in from tensorflow.python.eager import context File "C:\Users\Kaiwalya\Anaconda3\envs\py38\lib\site-packages\tensorflow\python\eager\context.py", line 35, in from tensorflow.python.client import pywrap_tf_session File "C:\Users\Kaiwalya\Anaconda3\envs\py38\lib\site-packages\tensorflow\python\client\pywrap_tf_session.py", line 19, in from tensorflow.python.client._pywrap_tf_session import * ImportError: SystemError: <built-in method contains__ of dict object at 0x00000249DEF7AFC0> returned a result with an error set

Kannan665 commented 2 years ago

I remember, had the same issue while trying it on Windows......When I see the error messages, I am able to understood, you are not using any GPU......Can you post the pip list of your conda environment and I will check with my environment and check, if there are any incompatibile libraries..... I changed to docker and I am running on Ubuntu the same TF2 OD API and it took some time to settle......But running all these using docker is just amazing.......

kaiwalya4850 commented 2 years ago

Yes yes I am not using GPU, I just need a working framework on windows right now. Okay, so before coming to list of libraries, can you share that docker file? Maybe I can try it using WSL or so.

Also here's the list(have some extras too):

absl-py==1.0.0 affine==2.3.1 anyio==3.5.0 apache-beam==2.38.0 argon2-cffi==21.3.0 argon2-cffi-bindings==21.2.0 asgiref==3.5.1 asttokens==2.0.5 astunparse==1.6.3 attrs==21.4.0 avro-python3==1.10.2 backcall==0.2.0 beautifulsoup4==4.11.1 bleach==5.0.0 boto3==1.22.6 botocore==1.25.6 CacheControl==0.12.11 cachetools==5.0.0 certifi==2021.10.8 cffi==1.15.0 charset-normalizer==2.0.12 click==8.1.3 click-plugins==1.1.1 cligj==0.7.2 cloudpickle==2.0.0 colorama==0.4.4 contextlib2==21.6.0 crcmod==1.7 cycler==0.11.0 Cython==0.29.28 debugpy==1.6.0 decorator==5.1.1 defusedxml==0.7.1 dill==0.3.1.1 dm-tree==0.1.7 docopt==0.6.2 entrypoints==0.4 executing==0.8.3 fastapi==0.75.2 fastavro==1.4.11 fastjsonschema==2.15.3 firebase-admin==5.2.0 Flask==2.1.2 flatbuffers==2.0 fonttools==4.33.3 gast==0.5.3 gcloud==0.18.3 GDAL @ file:///C:/Users/Kaiwalya/Desktop/micasense_ga/imageprocessing/GDAL-3.4.2-cp38-cp38-win_amd64.whl gin-config==0.5.0 google-api-core==2.7.3 google-api-python-client==2.47.0 google-auth==2.6.6 google-auth-httplib2==0.1.0 google-auth-oauthlib==0.4.6 google-cloud-core==2.3.0 google-cloud-firestore==2.4.0 google-cloud-storage==2.3.0 google-crc32c==1.3.0 google-pasta==0.2.0 google-resumable-media==2.3.2 googleapis-common-protos==1.56.0 grpcio==1.46.1 grpcio-status==1.46.1 h11==0.13.0 h5py==3.6.0 hdfs==2.7.0 httplib2==0.19.1 idna==3.3 imageio==2.19.1 importlib-metadata==4.11.3 importlib-resources==5.7.1 ipykernel==6.13.0 ipython==8.3.0 ipython-genutils==0.2.0 ipywidgets==7.7.0 itsdangerous==2.1.2 jedi==0.18.1 Jinja2==3.1.2 jmespath==1.0.0 joblib==1.1.0 jsonschema==4.4.0 jupyter==1.0.0 jupyter-client==7.3.0 jupyter-console==6.4.3 jupyter-core==4.10.0 jupyterlab-pygments==0.2.2 jupyterlab-widgets==1.1.0 jws==0.1.3 kaggle==1.5.12 keras==2.8.0 Keras-Preprocessing==1.1.2 kiwisolver==1.4.2 libclang==14.0.1 lvis==0.5.3 lxml==4.8.0 Markdown==3.3.6 MarkupSafe==2.1.1 matplotlib==3.5.2 matplotlib-inline==0.1.3 micasense @ file:///C:/Users/Kaiwalya/Desktop/micasense_ga/imageprocessing mistune==0.8.4 msgpack==1.0.3 nbclient==0.6.2 nbconvert==6.5.0 nbformat==5.4.0 nest-asyncio==1.5.5 networkx==2.8 notebook==6.4.11 numpy==1.22.3 oauth2client==4.1.3 oauthlib==3.2.0 object-detection @ file:///C:/Users/Kaiwalya/Desktop/tf_ga/models/research opencv-python==4.5.5.64 opencv-python-headless==4.5.5.64 opt-einsum==3.3.0 orjson==3.6.8 packaging==21.3 pandas==1.4.2 pandas-datareader==0.10.0 pandocfilters==1.5.0 parso==0.8.3 pickleshare==0.7.5 Pillow==9.1.0 portalocker==2.4.0 prometheus-client==0.14.1 promise==2.3 prompt-toolkit==3.0.29 proto-plus==1.20.3 protobuf==3.20.1 psutil==5.9.0 pure-eval==0.2.2 py-cpuinfo==8.0.0 pyarrow==6.0.1 pyasn1==0.4.8 pyasn1-modules==0.2.8 pycocotools @ git+https://github.com/philferriere/cocoapi.git@2929bd2ef6b451054755dfd7ceb09278f935f7ad#subdirectory=PythonAPI pycparser==2.21 pycryptodome==3.14.1 pydantic==1.9.0 pydot==1.4.2 PyExifTool==0.4.13 Pygments==2.12.0 pymongo==3.12.3 pyparsing==2.4.7 Pyrebase==3.0.27 Pyrebase4==4.5.0 pyrsistent==0.18.1 pysolar==0.10 python-dateutil==2.8.2 python-jwt==2.0.1 python-slugify==6.1.2 pytz==2022.1 PyWavelets==1.3.0 pywin32==304 pywinpty==2.0.5 PyYAML==5.4.1 pyzbar==0.1.9 pyzmq==22.3.0 qtconsole==5.3.0 QtPy==2.1.0 rasterio @ file:///C:/Users/Kaiwalya/Desktop/micasense_ga/imageprocessing/rasterio-1.2.10-cp38-cp38-win_amd64.whl regex==2022.4.24 requests==2.27.1 requests-oauthlib==1.3.1 requests-toolbelt==0.9.1 rsa==4.8 s3transfer==0.5.2 sacrebleu==2.0.0 scikit-image==0.19.2 scikit-learn==1.0.2 scipy==1.8.0 Send2Trash==1.8.0 sentencepiece==0.1.96 seqeval==1.2.2 six==1.16.0 sklearn==0.0 sniffio==1.2.0 snuggs==1.4.7 soupsieve==2.3.2.post1 stack-data==0.2.0 starlette==0.17.1 tabulate==0.8.9 tensorboard==2.8.0 tensorboard-data-server==0.6.1 tensorboard-plugin-wit==1.8.1 tensorflow==2.8.0 tensorflow-addons==0.16.1 tensorflow-datasets==4.5.2 tensorflow-hub==0.12.0 tensorflow-io==0.25.0 tensorflow-io-gcs-filesystem==0.25.0 tensorflow-metadata==1.7.0 tensorflow-model-optimization==0.7.2 tensorflow-text==2.8.2 termcolor==1.1.0 terminado==0.13.3 text-unidecode==1.3 tf-estimator-nightly==2.8.0.dev2021122109 tf-models-official==2.8.0 tf-slim==1.1.0 threadpoolctl==3.1.0 tifffile==2022.5.4 tinycss2==1.1.1 tornado==6.1 tqdm==4.64.0 traitlets==5.1.1 typeguard==2.13.3 typing_extensions==4.2.0 uritemplate==4.1.1 urllib3==1.26.9 uvicorn==0.17.6 wcwidth==0.2.5 webencodings==0.5.1 Werkzeug==2.1.2 widgetsnbextension==3.6.0 wincertstore==0.2 wrapt==1.14.1 zipp==3.8.0

Kannan665 commented 2 years ago

TF-image-od.py from Armaan's post worked very well for me in one of my earlier installations of conda and when I tried it few weeks before, it was throwing all sorts of errors.....It is nothing to do with armaan's code, but it has to do with some incompatible libraries on my newer version of conda installation and Tensorflow.......I have to debug my installation to make armaan's code "TF-image-od.py" work......What I will do, is I will attach an alternate code for inferencing..... detect_objectsavi.zip

I will share you the arguments to be passed here.....I use Ubuntu and you have to suitably use the path in your windows os....

python detect_objects.py \ --model_path=/var/lib/tf_docker/workspace/training_demo/exported-models/my_ssd_mobilenet_v2_fpnlite_640x640_annotations_640_640_mito/saved_model \ --path_to_labelmap=/var/lib/tf_docker/workspace/training_demo/annotations_mito/label_map.pbtxt \ --images_dir=/var/lib/tf_docker/workspace/training_demo/images_mito/test \ --threshold=0.35 \ --class_ids="1"

I will share you the docker file and let me do some sanity check on it before sharing it to you.......I had tried it on WSL, but since it was not equal to bare metal performance and I was running low on C:\ drive memory and I preferred to run it in my ubuntu 18.04 LTS....... .