search

matterport / Mask_RCNN

Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow
Other
24.75k stars 11.71k forks source link

grey scale/RGBD training with pretrained weights #1753

Open ApoorvaSuresh opened 5 years ago

ApoorvaSuresh commented 5 years ago

are the pretrained weights used for all 4 channels? or just the RGB channels? How does it work for 1 channel images?

orestis-z commented 3 years ago

Hi, for a grayscale image you could substitute the first convolution layer (with depth 3) with a 2D convolution layer with weights from the mean of the original convolution layer. Here's a code snippet https://github.com/orestis-z/mask-rcnn-rgbd/blob/d590e0f5085f8cbe895a6698e284426fd0116aa4/instance_segmentation/sceneNet/train.py#L66-L85 for an example of how to extend the first convolution for RGB-D data by setting the weights of the depth channel to the mean of the RGB channels