Issue with following parameters in configs.py (TRAIN_SAVE_BEST_ONLY and TRAIN_SAVE_CHECKPOINT)

[Marcelbgit]

Hello Rokas,

thank you a lot for your tutorial. I followed your tutorial step by step exactly but go this error although I have changed the values and paths like you described.

Error: You don't have saved model weights to measure mAP, check TRAIN_SAVE_BEST_ONLY and TRAIN_SAVE_CHECKPOINT lines in configs.py

(tensorflow) marcel@marcel-engine:~/CV_AP/TensorFlow-2.x-YOLOv3$ python train.py 2021-02-05 09:53:31.114195: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1 2021-02-05 09:53:32.511502: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2021-02-05 09:53:32.537592: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 2194895000 Hz 2021-02-05 09:53:32.537992: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x561472bd5b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2021-02-05 09:53:32.538033: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2021-02-05 09:53:32.544174: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1 2021-02-05 09:53:32.555015: E tensorflow/stream_executor/cuda/cuda_driver.cc:314] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected 2021-02-05 09:53:32.555069: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (marcel-engine): /proc/driver/nvidia/version does not exist [name: "/device:CPU:0" device_type: "CPU" memory_limit: 268435456 locality { } incarnation: 14568546309647675352 , name: "/device:XLA_CPU:0" device_type: "XLA_CPU" memory_limit: 17179869184 locality { } incarnation: 10834350452177837327 physical_device_desc: "device: XLA_CPU device" ] GPUs [] skipping conv2d_74 skipping conv2d_66 skipping conv2d_58 configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model configure TEST options to validate model You don't have saved model weights to measure mAP, check TRAIN_SAVE_BEST_ONLY and TRAIN_SAVE_CHECKPOINT lines in configs.py My configs.py looks like this

**#================================================================ #

File name : configs.py

Author : PyLessons

Created date: 2020-08-18

Website : https://pylessons.com/

GitHub : https://github.com/pythonlessons/TensorFlow-2.x-YOLOv3

Description : yolov3 configuration file

#

================================================================

YOLO options

YOLO_TYPE = "yolov3" # yolov4 or yolov3 YOLO_FRAMEWORK = "tf" # "tf" or "trt" YOLO_V3_WEIGHTS = "model_data/yolov3.weights" YOLO_V4_WEIGHTS = "model_data/yolov4.weights" YOLO_V3_TINY_WEIGHTS = "model_data/yolov3-tiny.weights" YOLO_V4_TINY_WEIGHTS = "model_data/yolov4-tiny.weights" YOLO_TRT_QUANTIZE_MODE = "INT8" # INT8, FP16, FP32 YOLO_CUSTOM_WEIGHTS = False # "checkpoints/yolov3_custom" # used in evaluate_mAP.py and custom model detection, if not using leave False

YOLO_CUSTOM_WEIGHTS also used with TensorRT and custom model detection

YOLO_COCO_CLASSES = "model_data/coco/coco.names" YOLO_STRIDES = [8, 16, 32] YOLO_IOU_LOSS_THRESH = 0.5 YOLO_ANCHOR_PER_SCALE = 3 YOLO_MAX_BBOX_PER_SCALE = 100 YOLO_INPUT_SIZE = 416 if YOLO_TYPE == "yolov4": YOLO_ANCHORS = [[[12, 16], [19, 36], [40, 28]], [[36, 75], [76, 55], [72, 146]], [[142,110], [192, 243], [459, 401]]] if YOLO_TYPE == "yolov3": YOLO_ANCHORS = [[[10, 13], [16, 30], [33, 23]], [[30, 61], [62, 45], [59, 119]], [[116, 90], [156, 198], [373, 326]]]

Train options

TRAIN_YOLO_TINY = False TRAIN_SAVE_BEST_ONLY = True # saves only best model according validation loss (True recommended) TRAIN_SAVE_CHECKPOINT = False # saves all best validated checkpoints in training process (may require a lot disk space) (False recommended) TRAIN_CLASSES = "./model_data/license_plate_names.txt"#"mnist/mnist.names" TRAIN_ANNOT_PATH = "./model_data/license_plate_train.txt"#"mnist/mnist_train.txt" TRAIN_LOGDIR = "log" TRAIN_CHECKPOINTS_FOLDER = "checkpoints" TRAIN_MODEL_NAME = f"{YOLO_TYPE}_custom" TRAIN_LOAD_IMAGES_TO_RAM = True # With True faster training, but need more RAM TRAIN_BATCH_SIZE = 4 TRAIN_INPUT_SIZE = 416 TRAIN_DATA_AUG = True TRAIN_TRANSFER = True TRAIN_FROM_CHECKPOINT = False # "checkpoints/yolov3_custom" TRAIN_LR_INIT = 1e-4 TRAIN_LR_END = 1e-6 TRAIN_WARMUP_EPOCHS = 2 TRAIN_EPOCHS = 100

TEST options

TEST_ANNOT_PATH = "./model_data/license_plate_test.txt"#"mnist/mnist_test.txt" TEST_BATCH_SIZE = 4 TEST_INPUT_SIZE = 416 TEST_DATA_AUG = True TEST_DECTECTED_IMAGE_PATH = "" TEST_SCORE_THRESHOLD = 0.3 TEST_IOU_THRESHOLD = 0.45

YOLOv3-TINY and YOLOv4-TINY WORKAROUND

if TRAIN_YOLO_TINY: YOLO_STRIDES = [16, 32, 64]
YOLO_ANCHORS = [[[10, 14], [23, 27], [37, 58]], [[81, 82], [135, 169], [344, 319]], [[0, 0], [0, 0], [0, 0]]]**

Hopefully you can help me out! Best regards Marcel

[pythonlessons]

You are sure you got this file? TEST_ANNOT_PATH = "./model_data/license_plate_test.txt"#"mnist/mnist_test.txt"

[Marcelbgit]

Hey there,

thanks a lot for reply I found the problem about the folder structure. Now I am conducting the training with CPU. But I have another question to you. Hopefully you can help me with this too. I have already installed the necessary nvidia driver and are able to switch between my intel and nvidia graphic card. But I neither can use it for training (look device error above) nor are able to display it with the command nvidia-smi. Do you know a good nvidia driver instruction without purge and destroy the already setup. I tried to install the driver for training for several times without success.

[fitim-halimi]

Hello there, I'm also receiving this issue "You don't have saved model weights to measure mAP, check TRAIN_SAVE_BEST_ONLY and TRAIN_SAVE_CHECKPOINT lines in configs.py" also while training it shows this : configure TEST options to validate model

I would appreciate some help

[DextroLaev]

"You don't have saved model weights to measure mAP, check TRAIN_SAVE_BEST_ONLY and TRAIN_SAVE_CHECKPOINT lines in configs.py"

How to solve this issue ?