Open matiqul opened 6 years ago
Currently in the process of testing this, but if I understand correctly:
coco.py
with class_ids=[43, 77]
(no need to include background class);NUM_CLASSES = 3
again in coco.py
; thenpython coco.py train --dataset=/path/to/data --model=imagenet
; orcoco.py
on line 469, adding exclude=['mrcnn_class_logits', 'mrcnn_bbox_fc', 'mrcnn_bbox', 'mrcnn_mask']
to the load_weights
call. Then run using python coco.py train --dataset=/path/to/data --model=coco
Do you mean: model.load_weights(model_path, by_name=True, exclude=['mrcnn_class_logits', 'mrcnn_bbox_fc', 'mrcnn_bbox', 'mrcnn_mask'])
right?
And do you know how to do python coco.py train --dataset=/path/to/data --model=coco
using the ROS version of this code? Should it be somehow integrated into the launch file?https://github.com/akio/mask_rcnn_ros
Yes, correct on model.load_weights
.
I've not used the ROS version, but I should think the following would work:
COCO_MODEL_PATH
to point to your new weights file, and CLASS_NAMES
to be your reduced set of objects.in addition with training three classes only, i want to draw the prediction box or mask on three class only. how to do that. in other word i can say, let the model detect all the class but while sowing it should draw box over this three classes only. please help me to do that?
@paolaforte there is no need to change the num_classes in config.py. Because when you set num_classes in coco.py, you rewirte it.
@paolaforte how did you know the [0,43,77] is the (BG, fork, scissors) ? where can i find the class_id? THX
@anju54 I have the same issue, did you happen to get a solution to your query ? @paolaforte class names displayed are in order (starting from 0) hence the mapping 0-80 (total 81)
Yeah what u want to know. I have executed this algorithm successfully
@anju54 I don't want to retrain the model but just want to display selected classes. Can you please post your changes ?
@waleedka I was working with this https://github.com/Tony607/colab-mask-rcnn/blob/master/Colab_Mask_R_CNN_Demo.ipynb I can run this successfully using 80 classes, but having trouble changing no of classes to just 30.
The code runs after including "exclude= ..." but it It result jpg's have classe ID's that I have not included, like [cow, tie etc..]. Not sure what is going wrong here.
I had to modify the local samples/coco.py file include only related classes:
def load_coco(self, dataset_dir, subset, year=DEFAULT_DATASET_YEAR, class_ids=[0,1,2,3,4,5,6,7,8,10,11,12,13,14,15,26,27,58,59,60,61,62,63,64,70,71,72,73,74,78],
class_map=None, return_coco=False, auto_download=False):
...
Below is my notebook snippet:
# Import COCO config
sys.path.append(os.path.join(ROOT_DIR, "samples/coco/")) # To find local version
print(ROOT_DIR)
import coco
#We use a K80 GPU with 24GB memory, which can fit 3 images.
batch_size = 5
ROOT_DIR = os.getcwd()
MODEL_DIR = os.path.join(ROOT_DIR, "logs")
VIDEO_DIR = os.path.join(ROOT_DIR, "videos")
VIDEO_SAVE_DIR = os.path.join(VIDEO_DIR, "save_2")
COCO_MODEL_PATH = os.path.join(ROOT_DIR, "mask_rcnn_coco.h5")
if not os.path.exists(COCO_MODEL_PATH):
utils.download_trained_weights(COCO_MODEL_PATH)
class CocoConfig(Config):
"""Configuration for training on MS COCO.
Derives from the base Config class and overrides values specific
to the COCO dataset.
"""
# Give the configuration a recognizable name
NAME = "coco"
# We use a GPU with 12GB memory, which can fit two images.
# Adjust down if you use a smaller GPU.
IMAGES_PER_GPU = 1
# Uncomment to train on 8 GPUs (default is 1)
# GPU_COUNT = 8
# Number of classes (including background)
#NUM_CLASSES = 37 # COCO has 80 classes
NUM_CLASSES = 30
#class InferenceConfig(coco.CocoConfig):
class InferenceConfig(CocoConfig):
# Set batch size to 1 since we'll be running inference on
# one image at a time. Batch size = GPU_COUNT * IMAGES_PER_GPU
GPU_COUNT = 1
IMAGES_PER_GPU = 5
#MAX_GT_INSTANCES = 100
#TRAIN_ROIS_PER_IMAGE = 50
#BACKBONE = "resnet50" #not working at all!
#RPN_ANCHOR_STRIDE = 2
#POST_NMS_ROIS_TRAINING = 1000
#POST_NMS_ROIS_INFERENCE = 500
#IMAGE_MIN_DIM = 400 #really much faster but bad results
#IMAGE_MAX_DIM = 512
#DETECTION_MAX_INSTANCES = 50 #a little faster but some instances not recognized
config = InferenceConfig()
config.display()
model = modellib.MaskRCNN(mode="inference", model_dir=MODEL_DIR, config=config)
model.load_weights(COCO_MODEL_PATH, by_name=True,exclude=['mrcnn_class_logits', 'mrcnn_bbox_fc', 'mrcnn_bbox', 'mrcnn_mask'])
if you simply want to draw that prediction bounding box over class = 30 then just modify ur visualize.py file In my case, I wanted it for only person class so I fixed it for class = 1 in visualize.py. here is the modification in visualize.py ( only in display_instances() method)
def display_instances(image, boxes, masks, class_ids, class_names, scores=None, title="", figsize=(16, 16), ax=None): """ boxes: [num_instance, (y1, x1, y2, x2, class_id)] in image coordinates. masks: [height, width, num_instances] class_ids: [num_instances] class_names: list of class names of the dataset scores: (optional) confidence scores for each box figsize: (optional) the size of the image. """
N = boxes.shape[0]
if not N:
print("\n*** No instances to display *** \n")
else:
assert boxes.shape[0] == masks.shape[-1] == class_ids.shape[0]
if not ax:
_, ax = plt.subplots(1, figsize=figsize)
# Generate random colors
colors = random_colors(N)
# Show area outside image boundaries.
height, width = image.shape[:2]
ax.set_ylim(height + 10, -10)
ax.set_xlim(-10, width + 10)
ax.axis('off')
ax.set_title(title)
masked_image = image.astype(np.uint32).copy()
t = []
t2 = []
t3 = []
t4 = []
for i in range(N):
color = colors[i]
class_id = class_ids[i]
# Bounding box
if not np.any(boxes[i]):
# Skip this instance. Has no bbox. Likely lost in image cropping.
continue
if (class_id == 1):
y1, x1, y2, x2 = boxes[i]
p = patches.Rectangle((x1, y1), x2 - x1, y2 - y1, linewidth=2,
alpha=0.7, linestyle="dashed",
edgecolor=color, facecolor='none')
ax.add_patch(p)
'''arr = [y1, x1, y2, x2]
a = np.array(arr)
# rows, cols = a.shape
t = np.append(t, a[0])
t2 = np.append(t2, a[1])
t3 = np.append(t3, a[2])
t4 = np.append(t4, a[3])
# print(t)'''
# Label
#class_id = class_ids[i]
#print(class_id)
score = scores[i] if scores is not None else None
label = class_names[class_id]
#print(label)
if (class_id == 1):
x = random.randint(x1, (x1 + x2) // 2)
caption = "{} {:.3f}".format(label, score) if score else label
ax.text(x1, y1 + 8, caption,
color='w', size=11, backgroundcolor="none")
# Mask
if (class_id == 1):
mask = masks[:, :, i]
masked_image = apply_mask(masked_image, mask, color)
# Mask Polygon
# Pad to ensure proper polygons for masks that touch image edges.
padded_mask = np.zeros(
(mask.shape[0] + 2, mask.shape[1] + 2), dtype=np.uint8)
padded_mask[1:-1, 1:-1] = mask
contours = find_contours(padded_mask, 0.5)
for verts in contours:
# Subtract the padding and flip (y, x) to (x, y)
verts = np.fliplr(verts) - 1
p = Polygon(verts, facecolor="none", edgecolor=color)
ax.add_patch(p)
'''array1 = [t, t2, t3, t4]
a1 = np.array(array1)
rr = np.transpose(a1)
d = pd.DataFrame(rr, columns=['y1', 'x1', 'y2', 'x2'])
d.to_csv('/home/anju/Documents/Mask_RCNN/a_f.csv', index=None)
print(a1)'''
ax.imshow(masked_image.astype(np.uint8))
plt.show()
How to train maskrcnn with both coco dataset and external dataset? I mean in terms of some specific classes, coco may be not enough for training.
Yeah what u want to know. I have executed this algorithm successfully
@anju54 You said you've executed this algorithm successfully, Your problem was also related to mine, so please have a look at my problem.
I'm doing a predictions for cotton plant field. I've three classes, (BG, Cotton, Weed). I don't want my model to detect or label anything else so my dataset
consist of 100 Cotton field images. Which has both cotton plant and weed plant in it, then I annotated them with VIA
tool. So I modified almost all code files including coco.py
model.py
config.py
I think I also modified utils.py
, and trained model. After so many bugs, and three weeks of debugging my model started training on google colab.
Above Image is copied from Inspect_plant.ipynb
which is actually modified form of inspect_ballon_data.ipynb
. But After training, When I started detection, I didn't got expected results. Below Images has splashed from my plant model.
It has gray-scaled whole image and returned original color pixel for detected classes, It also has significant accuracy issue, It is splashing cotton
accurately but neglecting weed
most of times. My major concern is weed
, I need to get all weed plant
coordinates from images, because I'll then drive robotic arm to pluck weeds from field.
Now I want it to return results as it did in above inspect_plant.ipynb
image, that I've attached above.
Kindly help me with step-wise instruction to reach my target. @waleedka I've seen your name in every code file, kindly reply me to get into this.
Currently in the process of testing this, but if I understand correctly:
- Edit
coco.py
withclass_ids=[43, 77]
(no need to include background class);- Edit
NUM_CLASSES = 3
again incoco.py
; thenEither:
- Start training from the imagenet weights, i.e.
python coco.py train --dataset=/path/to/data --model=imagenet
; or- To train from the pre-trained COCO weights, edit
coco.py
on line 469, addingexclude=['mrcnn_class_logits', 'mrcnn_bbox_fc', 'mrcnn_bbox', 'mrcnn_mask']
to theload_weights
call. Then run usingpython coco.py train --dataset=/path/to/data --model=coco
does this really help. i trained the model use cocodataset but only person classes. but when I evaluate the model,I got all the eval ap/map are nums
following the steps of #161 and #104.