Improve models documentation

[luxedo]

As I understand, you can point to both Mask RCNN and DEXTR models using environment variables.

I managed to make DEXTR work, but not Mask RCNN. I'm getting the following warning:

[WARNING] MaskRCNN model is disabled

Even though I'm pointing to the model file with an MASK_RCNN_FILE variable. I'd love to run my RCNN models in new images.

Can you provide extra documentation for the topic?

[jsbroks]

The environment values need to changed within the docker-compose.yml file. An example can be seen in the docker-compose.dev.yml, where I have it configured to us the pre-trained maskrcnn coco model.

In a general use case:

1. Set MASK_RCNN_FILE to point to your h5 file (e.g /models/your-h5-file.h5).
2. Set MASK_RCNN_CLASSES to your classes which should be in the order in which your models index's as output

   Theses environment variables should be set in the docker-compose.yml

The model should now load on boot. Next we need to set the datasets url to point to this model.

1. Inside the annotator click the settings icon on the right panel.
2. type the url of the model, if using the built in mask rcnn its /api/model/maskrcnn

[luxedo]

It worked flawlessly! Thank you very much.

I was setting the default value in config.py not in the compose file.

[NicksonYap]

@jsbroks

I was looking for the wiki guide to run Mask RCNN model and can't find it I suppose I'll use this as the guide? :sweat_smile:

[NicksonYap]

In response to: https://github.com/jsbroks/coco-annotator/issues/204#issuecomment-492835372 (Looking for the MaskRCNN interface)

I wasn't able to find any interface related to MaskRCNN I'm at v0.10.5, master branch

My steps:

In a general use case:

1. Set MASK_RCNN_FILE to point to your h5 file (e.g /models/your-h5-file.h5).
2. Set MASK_RCNN_CLASSES to your classes which should be in the order in which your models index's as output

docker-compose.dev.yml:

docker-compose.yml:

I ran:

docker-compose -f docker-compose.dev.yml up --build

In console, it shows the model is loaded

[2019-05-21 06:21:02 +0000] [10] [INFO] Loaded MaskRCNN model: /models/mask_rcnn_zuppa_0149.h5

The webpage also loads successfully, after increasing gunicorn timeout https://github.com/jsbroks/coco-annotator/issues/199#issuecomment-487707981

The model should now load on boot. Next we need to set the datasets url to point to this model.

1. Inside the annotator click the settings icon on the right panel.
2. type the url of the model, if using the built in mask rcnn its /api/model/maskrcnn

The settings icon on the right doesn't seem relevant at all

So I tried the one on the left, and doesn't seem to have any effect

I have dug the source code, and no Vue files are using the /api/model/maskrcnn endpoint

What was I missing?

Edit: Turns out you need to click this button after adding the Annotate URL :)

PS:

Remember to add your Categories, they must be the same name as the Classes of the model

[xden2331]

I tried modifying the docker-compose.yml as specified:

but get this warning when I try docker-compose up:

Thanks for any assistance!

The value of the environment is the path in the container, not the path in the host. You can see under the volumes key, you have linked the container dir to the host dir. The solution is: change the MASK_RCNN_FILE value from "/U../a.../c.../models/...h5" to "/models/...h5"

[Marwen-Bhj]

it literally doesn't do anything for me, I followed the exact same steps, having my classes as : person, motorcycle. I click on that cloud icon and nothing happens and in the console only outputs this : [DEBUG] POST /api/model/maskrcnn Just to be sure, when I click on annotate it should detect person and motorcycle within the image and mask them right ?

[raghavgurbaxani]

Hi I am also having the same issue, I am able to get dexter to work but not Mask RCNN, can you help out @jsbroks ?

I made the changes to the docker compose file but when I click the annotate button , nothing happens....

[TY48]

I had the same issue as @Marwen-Bhj and @raghavgurbaxani using the default pretrained model, and made it work after some debugging. Now it detects objects when I click "Annotate Image" button and display the result.

Solution

Add ALL the predefined classes in MASK_RCNN_CLASSES (just like docker-compose.dev.yml), not only a few of them.

docker-compose.yml

services:
  webserver:
    image: jsbroks/coco-annotator:webserver-stable
    container_name: annotator_webclient
    restart: always
    ports:
      - "5000:5000"
    environment:
      - SECRET_KEY=RandomSecretKeyHere
      - FILE_WATCHER=true
      - MASK_RCNN_FILE=/models/mask_rcnn_coco.h5
      - |
        MASK_RCNN_CLASSES=
        BG,person,bicycle,car,motorcycle,airplane,
        bus,train,truck,boat,traffic light,
        fire hydrant,stop sign,parking meter,bench,bird,
        cat,dog,horse,sheep,cow,elephant,bear,
        zebra,giraffe,backpack,umbrella,handbag,tie,
        suitcase,frisbee,skis,snowboard,sports ball,
        kite,baseball bat,baseball glove,skateboard,
        surfboard,tennis racket,bottle,wine glass,cup,
        fork,knife,spoon,bowl,banana',apple,
        sandwich,orange,broccoli,carrot,hot dog,pizza,
        donut,cake,chair,couch,potted plant,bed,
        dining table,toilet,tv,laptop,mouse,remote,
        keyboard,cell phone,microwave,oven,toaster,
        sink,refrigerator,book,clock,vase,scissors,
        teddy bear,hair drier,toothbrush

Cause

The model was not loaded correctly at the line self.model.load_weights(COCO_MODEL_PATH, by_name=True) (mask_rcnn.py in this repository). This line was raising an error ValueError: Layer #389 (named "mrcnn_bbox_fc"), weight <tf.Variable 'mrcnn_bbox_fc/kernel:0' shape=(1024, 8) dtype=float32_ref> has shape (1024, 8), but the saved weight has shape (1024, 324).. Since this error is not caught/logged, I had to docker exec into the container to find it. This issue is discussed in https://github.com/matterport/Mask_RCNN/issues/849 with some solutions.