pythonlessons / TensorFlow-2.x-YOLOv3

YOLOv3 implementation in TensorFlow 2.3.1
https://pylessons.com/
MIT License
605 stars 327 forks source link

cant run #53

Closed hdihd9162 closed 4 years ago

hdihd9162 commented 4 years ago

i train a dataset but cant run custom

File "/home/pi/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 2785, in _is_hdf5_filepath
    return (filepath.endswith('.h5') or filepath.endswith('.keras') or
AttributeError: 'bool' object has no attribute 'endswith'
pythonlessons commented 4 years ago

Not enough information to tell whats wrong on your side

hdihd9162 commented 4 years ago

i dont have problem with train train is work with out error but when i run detection_custom.py this error show python3 .\detection_custom.py 2020-09-16 12:45:21.159549: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found 2020-09-16 12:45:21.159735: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. 2020-09-16 12:45:24.279950: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library nvcuda.dll 2020-09-16 12:45:24.320347: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: pciBusID: 0000:01:00.0 name: GeForce GTX 1650 computeCapability: 7.5 coreClock: 1.56GHz coreCount: 16 deviceMemorySize: 4.00GiB deviceMemoryBandwidth: 119.24GiB/s 2020-09-16 12:45:24.321221: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found 2020-09-16 12:45:24.321845: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cublas64_10.dll'; dlerror: cublas64_10.dll not found 2020-09-16 12:45:24.322412: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cufft64_10.dll'; dlerror: cufft64_10.dll not found 2020-09-16 12:45:24.323040: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'curand64_10.dll'; dlerror: curand64_10.dll not found 2020-09-16 12:45:24.323642: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cusolver64_10.dll'; dlerror: cusolver64_10.dll not found 2020-09-16 12:45:24.324092: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cusparse64_10.dll'; dlerror: cusparse64_10.dll not found 2020-09-16 12:45:24.324537: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cudnn64_7.dll'; dlerror: cudnn64_7.dll not found 2020-09-16 12:45:24.324635: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1753] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform. Skipping registering GPU devices... 2020-09-16 12:45:24.327576: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations: AVX2 To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2020-09-16 12:45:24.342091: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x28c3effd770 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-09-16 12:45:24.342567: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2020-09-16 12:45:24.345179: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix: 2020-09-16 12:45:24.345841: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263] Traceback (most recent call last): File ".\detection_custom.py", line 30, in <module> yolo.load_weights(YOLO_CUSTOM_WEIGHTS) # use custom weights File "C:\Users\Asus\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\tensorflow\python\keras\engine\training.py", line 2172, in load_weights if _is_hdf5_filepath(filepath): File "C:\Users\Asus\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\tensorflow\python\keras\engine\training.py", line 2785, in _is_hdf5_filepath return (filepath.endswith('.h5') or filepath.endswith('.keras') or AttributeError: 'bool' object has no attribute 'endswith'

hdihd9162 commented 4 years ago

can you send to me this tutorial https://pylessons.com/YOLOv3-TF2-custrom-images/ final code??

pythonlessons commented 4 years ago

show me your's configs.py file final code is on github

hdihd9162 commented 4 years ago

` YOLO_TYPE = "yolov3" # yolov4 or yolov3 YOLO_FRAMEWORK = "tf" # "tf" or "trt" YOLO_V3_WEIGHTS = "model_data/yolov3.weights" YOLO_V4_WEIGHTS = "model_data/yolov4.weights" YOLO_V3_TINY_WEIGHTS = "model_data/yolov3-tiny.weights" YOLO_V4_TINY_WEIGHTS = "model_data/yolov4-tiny.weights" YOLO_TRT_QUANTIZE_MODE = "INT8" # INT8, FP16, FP32 YOLO_CUSTOM_WEIGHTS = False # "checkpoints/yolov3_custom" # used in evaluate_mAP.py and custom model detection, if not using leave False

YOLO_CUSTOM_WEIGHTS also used with TensorRT and custom model detection

YOLO_COCO_CLASSES = "model_data/coco/coco.names" YOLO_STRIDES = [8, 16, 32] YOLO_IOU_LOSS_THRESH = 0.5 YOLO_ANCHOR_PER_SCALE = 3 YOLO_MAX_BBOX_PER_SCALE = 100 YOLO_INPUT_SIZE = 416 if YOLO_TYPE == "yolov4": YOLO_ANCHORS = [[[12, 16], [19, 36], [40, 28]], [[36, 75], [76, 55], [72, 146]], [[142,110], [192, 243], [459, 401]]] if YOLO_TYPE == "yolov3": YOLO_ANCHORS = [[[10, 13], [16, 30], [33, 23]], [[30, 61], [62, 45], [59, 119]], [[116, 90], [156, 198], [373, 326]]]

Train options

TRAIN_YOLO_TINY = False TRAIN_SAVE_BEST_ONLY = True # saves only best model according validation loss (True recommended) TRAIN_SAVE_CHECKPOINT = False # saves all best validated checkpoints in training process (may require a lot disk space) (False recommended) TRAIN_CLASSES = "./model_data/license_plate_names.txt" TRAIN_ANNOT_PATH = "./model_data/license_plate_train.txt" TRAIN_LOGDIR = "log" TRAIN_CHECKPOINTS_FOLDER = "checkpoints" TRAIN_MODEL_NAME = f"{YOLO_TYPE}_custom" TRAIN_LOAD_IMAGES_TO_RAM = True # With True faster training, but need more RAM TRAIN_BATCH_SIZE = 4 TRAIN_INPUT_SIZE = 416 TRAIN_DATA_AUG = True TRAIN_TRANSFER = True TRAIN_FROM_CHECKPOINT = False # "checkpoints/yolov3_custom" TRAIN_LR_INIT = 1e-4 TRAIN_LR_END = 1e-6 TRAIN_WARMUP_EPOCHS = 2 TRAIN_EPOCHS = 100

TEST options

TEST_ANNOT_PATH = "./model_data/license_plate_test.txt" TEST_BATCH_SIZE = 4 TEST_INPUT_SIZE = 416 TEST_DATA_AUG = True TEST_DECTECTED_IMAGE_PATH = "" TEST_SCORE_THRESHOLD = 0.3 TEST_IOU_THRESHOLD = 0.45

YOLOv3-TINY and YOLOv4-TINY WORKAROUND

if TRAIN_YOLO_TINY: YOLO_STRIDES = [16, 32, 64] YOLO_ANCHORS = [[[10, 14], [23, 27], [37, 58]], [[81, 82], [135, 169], [344, 319]], [[0, 0], [0, 0], [0, 0]]]

`

hdihd9162 commented 4 years ago

i get this error after some change .\detection_custom.py 2020-09-16 13:39:28.859256: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found 2020-09-16 13:39:28.860530: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. 2020-09-16 13:39:39.148535: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library nvcuda.dll 2020-09-16 13:39:39.263881: E tensorflow/stream_executor/cuda/cuda_driver.cc:314] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected 2020-09-16 13:39:39.274030: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: hdihd-lenovo 2020-09-16 13:39:39.274959: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: hdihd-lenovo 2020-09-16 13:39:39.278957: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations: AVX2 To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2020-09-16 13:39:39.330101: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1d9e46ffb90 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-09-16 13:39:39.330330: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version Traceback (most recent call last): File ".\detection_custom.py", line 19, in <module> yolo.load_weights("./checkpoints/yolov3_custom") File "C:\Users\Asus\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\tensorflow\python\keras\engine\training.py", line 2182, in load_weights status = self._trackable_saver.restore(filepath, options) File "C:\Users\Asus\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\tensorflow\python\training\tracking\util.py", line 1319, in restore base.CheckpointPosition( File "C:\Users\Asus\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\tensorflow\python\training\tracking\base.py", line 209, in restore restore_ops = trackable._restore_from_checkpoint_position(self) # pylint: disable=protected-access File "C:\Users\Asus\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\tensorflow\python\training\tracking\base.py", line 913, in _restore_from_checkpoint_position current_position.checkpoint.restore_saveables( File "C:\Users\Asus\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\tensorflow\python\training\tracking\util.py", line 296, in restore_saveables new_restore_ops = functional_saver.MultiDeviceSaver( File "C:\Users\Asus\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\tensorflow\python\training\saving\functional_saver.py", line 340, in restore restore_ops = restore_fn() File "C:\Users\Asus\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\tensorflow\python\training\saving\functional_saver.py", line 316, in restore_fn restore_ops.update(saver.restore(file_prefix, options)) File "C:\Users\Asus\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\tensorflow\python\training\saving\functional_saver.py", line 110, in restore restore_ops[saveable.name] = saveable.restore( File "C:\Users\Asus\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\tensorflow\python\training\saving\saveable_object_util.py", line 126, in restore return resource_variable_ops.shape_safe_assign_variable_handle( File "C:\Users\Asus\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\tensorflow\python\ops\resource_variable_ops.py", line 311, in shape_safe_assign_variable_handle shape.assert_is_compatible_with(value_tensor.shape) File "C:\Users\Asus\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\tensorflow\python\framework\tensor_shape.py", line 1134, in assert_is_compatible_with raise ValueError("Shapes %s and %s are incompatible" % (self, other)) ValueError: Shapes (255,) and (18,) are incompatible

pythonlessons commented 4 years ago

I just updated detection_custom.py file, custom classes were missing, try it now. Actually right now detection_demo.py should also work with custom model.

Also, link your model name in the following line: YOLO_CUSTOM_WEIGHTS = "checkpoints/yolov3_custom" # used in evaluate_mAP.py and custom model detection, if not using leave False

hdihd9162 commented 4 years ago

i use this and no error ` import os os.environ['CUDA_VISIBLE_DEVICES'] = '0' import cv2 import numpy as np import tensorflow as tf from tensorflow.python.saved_model import tag_constants from yolov3.yolov4 import Create_Yolo from yolov3.utils import load_yolo_weights, detect_image, detect_video, detect_realtime from yolov3.configs import *

image_path = "./IMAGES/plate_2.jpg" video_path = "./IMAGES/test.mp4" YOLO_CUSTOM_WEIGHTS = "checkpoints/yolov3_custom"

if YOLO_FRAMEWORK == "tf": # TensorFlow detection if YOLO_TYPE == "yolov4": Darknet_weights = YOLO_V4_TINY_WEIGHTS if TRAIN_YOLO_TINY else YOLO_V4_WEIGHTS if YOLO_TYPE == "yolov3": Darknet_weights = YOLO_V3_TINY_WEIGHTS if TRAIN_YOLO_TINY else YOLO_V3_WEIGHTS yolo = Create_Yolo(input_size=YOLO_INPUT_SIZE, CLASSES=TRAIN_CLASSES) yolo.load_weights(YOLO_CUSTOM_WEIGHTS) # use custom weights

elif YOLO_FRAMEWORK == "trt": # TensorRT detection saved_model_loaded = tf.saved_model.load(YOLO_CUSTOM_WEIGHTS, tags=[tag_constants.SERVING]) signature_keys = list(saved_model_loaded.signatures.keys()) yolo = saved_model_loaded.signatures['serving_default']

detect_image(yolo, image_path, "./IMAGES/plate_1_detect.jpg", input_size=YOLO_INPUT_SIZE, show=True, CLASSES=TRAIN_CLASSES, rectangle_colors=(255,0,0))

detect_video(yolo, video_path, './IMAGES/detected.mp4', input_size=YOLO_INPUT_SIZE, show=False, CLASSES=TRAIN_CLASSES, rectangle_colors=(255,0,0))

detect_realtime(yolo, '', input_size=YOLO_INPUT_SIZE, show=True, CLASSES=TRAIN_CLASSES, rectangle_colors=(255, 0, 0))

` if i need to show object lable in terminal can you help me?

pythonlessons commented 4 years ago

go to detect_video function, use the print function to print what you need

hdihd9162 commented 4 years ago

can you send a example??

pythonlessons commented 4 years ago

It's an easy task, you should do this by your self to learn coding. Life is hard, no one will give you final solutions every time, you should learn doing something by yourself. I wrote this code quite simply so that everyone could understand with a small amount of effort, so, please...

Myron1996 commented 3 years ago

I cant run the custom_detection.py . Its unable to find weights. I am actually trying to run with custom checkpoints file.

2020-10-16 14:53:17.139938: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-10-16 14:53:17.176233: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties:
pciBusID: 0000:68:00.0 name: TITAN RTX computeCapability: 7.5
coreClock: 1.77GHz coreCount: 72 deviceMemorySize: 23.65GiB deviceMemoryBandwidth: 625.94GiB/s
2020-10-16 14:53:17.176746: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-10-16 14:53:17.179093: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-10-16 14:53:17.181202: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-10-16 14:53:17.181582: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-10-16 14:53:17.183482: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-10-16 14:53:17.184442: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-10-16 14:53:17.188382: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-10-16 14:53:17.189988: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0
Traceback (most recent call last):
  File "counting.py", line 73, in <module>
    Darknet_weights = YOLO_DARKNET_WEIGHTS
NameError: name 'YOLO_DARKNET_WEIGHTS' is not defined

config.py

#================================================================
#
#   File name   : configs.py
#   Author      : PyLessons
#   Created date: 2020-08-18
#   Website     : https://pylessons.com/
#   GitHub      : https://github.com/pythonlessons/TensorFlow-2.x-YOLOv3
#   Description : yolov3 configuration file
#
#================================================================

# YOLO options
YOLO_TYPE                   = "yolov3" # yolov4 or yolov3
YOLO_FRAMEWORK              = "tf" # "tf" or "trt"
YOLO_V3_WEIGHTS             = "./model_data/yolov3.weights"
YOLO_V4_WEIGHTS             = "model_data/yolov4.weights"
YOLO_V3_TINY_WEIGHTS        = "model_data/yolov3-tiny.weights"
YOLO_V4_TINY_WEIGHTS        = "model_data/yolov4-tiny.weights"
YOLO_TRT_QUANTIZE_MODE      = "INT8" # INT8, FP16, FP32
YOLO_CUSTOM_WEIGHTS         = "checkpoints/yolov3_custom" # used in evaluate_mAP.py and custom model detection, if not using leave False
                            # YOLO_CUSTOM_WEIGHTS also used with TensorRT and custom model detection
YOLO_COCO_CLASSES           = "model_data/coco/coco.names"
YOLO_STRIDES                = [8, 16, 32]
YOLO_IOU_LOSS_THRESH        = 0.5
YOLO_ANCHOR_PER_SCALE       = 3
YOLO_MAX_BBOX_PER_SCALE     = 100
YOLO_INPUT_SIZE             = 416
if YOLO_TYPE                == "yolov4":
    YOLO_ANCHORS            = [[[12,  16], [19,   36], [40,   28]],
                               [[36,  75], [76,   55], [72,  146]],
                               [[142,110], [192, 243], [459, 401]]]
if YOLO_TYPE                == "yolov3":
    YOLO_ANCHORS            = [[[10,  13], [16,   30], [33,   23]],
                               [[30,  61], [62,   45], [59,  119]],
                               [[116, 90], [156, 198], [373, 326]]]
# Train optionsi
TRAIN_YOLO_TINY             = False
TRAIN_SAVE_BEST_ONLY        = True # saves only best model according validation loss (True recommended)
TRAIN_SAVE_CHECKPOINT       = False # saves all best validated checkpoints in training process (may require a lot disk space) (False recommended)
TRAIN_CLASSES               = "./model_data/dataset.txt" #"./mnist/mnist.names"
TRAIN_ANNOT_PATH            = "./train.txt" #"./mnist/mnist_train.txt"
TRAIN_LOGDIR                = "./log"
TRAIN_CHECKPOINTS_FOLDER    = "./checkpoints"
TRAIN_MODEL_NAME            = "yolo_custom"
TRAIN_LOAD_IMAGES_TO_RAM    = True # With True faster training, but need more RAM
TRAIN_BATCH_SIZE            = 16
TRAIN_INPUT_SIZE            = 416
TRAIN_DATA_AUG              = True
TRAIN_TRANSFER              = True
TRAIN_FROM_CHECKPOINT       = False #"checkpoints/yolov3_custom"
TRAIN_LR_INIT               = 1e-4
TRAIN_LR_END                = 1e-6
TRAIN_WARMUP_EPOCHS         = 2
TRAIN_EPOCHS                = 1000

# TEST options
TEST_ANNOT_PATH             = "./test.txt" #"./mnist/mnist_test.txt"
TEST_BATCH_SIZE             = 16
TEST_INPUT_SIZE             = 416
TEST_DATA_AUG               = False
TEST_DECTECTED_IMAGE_PATH   = ""
TEST_SCORE_THRESHOLD        = 0.3
TEST_IOU_THRESHOLD          = 0.45

#YOLOv3-TINY and YOLOv4-TINY WORKAROUND
if TRAIN_YOLO_TINY:
    YOLO_STRIDES            = [16, 32, 64]    
    YOLO_ANCHORS            = [[[10,  14], [23,   27], [37,   58]],
                               [[81,  82], [135, 169], [344, 319]],
                               [[0,    0], [0,     0], [0,     0]]]
pythonlessons commented 3 years ago

there is no "counting.py", its your custom?

Myron1996 commented 3 years ago

HI @pythonlessons . ya I actually modified the Object tracking script. But its alright it is working now.

Thank you so much for your quick response. Really appreciate it.