Tracking Object with Grayscale Frame

[samchrist12]

I already have a YoloV4-Tiny model that trained on grayscale image. I change save_model.py input layer to

if FLAGS.grayscale:
    input_layer = tf.keras.layers.Input([FLAGS.input_size, FLAGS.input_size, 1])
  else:
    input_layer = tf.keras.layers.Input([FLAGS.input_size, FLAGS.input_size, 3])

on object_tracker.py I already tried to add some code to make the grayscale work

frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
frame = frame[:, :, np.newaxis]

but I got error like this

Traceback (most recent call last):
  File "object_tracker.py", line 304, in <module>
    app.run(main)
  File "/home/se790/anaconda3/envs/yolov4-gpu/lib/python3.7/site-packages/absl/app.py", line 300, in run
    _run_main(main, args)
  File "/home/se790/anaconda3/envs/yolov4-gpu/lib/python3.7/site-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "object_tracker.py", line 218, in main
    features = encoder(frame, bboxes)
  File "/home/se790/Stickearn/yolov4-deepsort/tools/generate_detections.py", line 118, in encoder
    return image_encoder(image_patches, batch_size)
  File "/home/se790/Stickearn/yolov4-deepsort/tools/generate_detections.py", line 99, in __call__
    {self.input_var: data_x}, out, batch_size)
  File "/home/se790/Stickearn/yolov4-deepsort/tools/generate_detections.py", line 23, in _run_in_batches
    out[s:e] = f(batch_data_dict)
  File "/home/se790/Stickearn/yolov4-deepsort/tools/generate_detections.py", line 98, in <lambda>
    lambda x: self.session.run(self.output_var, feed_dict=x),
  File "/home/se790/anaconda3/envs/yolov4-gpu/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 958, in run
    run_metadata_ptr)
  File "/home/se790/anaconda3/envs/yolov4-gpu/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1157, in _run
    (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (1, 128, 64) for Tensor 'images:0', which has shape '(None, 128, 64, 3)'

any ideas on how to do the grayscale inferencing ? Thanks

[anandkoirala1]

@samchrist12 @theAIGuysCode For grayscale detection (model trained for grayscale) of RGB video and also useful for custom class names. All tested.. Hope it helps..

1. save_model.py file change input_size flag from 416 to your custom darknet model size or provide as option/parameter when compiling the .weights file to .pb files
2. save_model.py file change channels to be 1 as in picture 'my code'
3. /core/config.py file change __C.YOLO.CLASSES to provide path to your custom trained darknet model's '.names' file
4. object_tracker.py file change the size flag from 416 to the size used by your custom model that you compiled using save_model.py
5. change object_tracker.py file as the lines with 'my code' as in picture
6. Do not forget to re-compile tensorflow model using save_model.py after making above changes before using the object_tracker.py

[MilersWang]

@samchrist12 Hi Did you slove the problem? I have same problem with you. I wanna use RGBD(4 channels image). But it shows me this: ValueError: Cannot feed value of shape (1, 128, 64, 4) for Tensor 'images:0', which has shape '(None, 128, 64, 3)'

Thanks~

[MilersWang]

@anandkoirala1 I already use your method, but it didn't work.

It shows this: ValueError: Cannot feed value of shape (1, 128, 64, 4) for Tensor 'images:0', which has shape '(None, 128, 64, 3)' It seems like model already change to 4 channels. And it can predict well. It can produce bboxes.

In this step, I failed:

features = encoder(frame, bboxes)

It seems like deepSORT do not provide different channels tracking.

In line 52&53:

model_filename = 'model_data/mars-small128.pb'
encoder = gdet.create_box_encoder(model_filename, batch_size=1)

Maybe i need to change this!