Gender model always predict the same class with same probability

jplu commented 8 years ago

Hello,

I'm currently testing your tool for gender recognition. I set up a gender service with the model provided here: http://www.deepdetect.com/models/gender.tar.bz2

And every time I try to predict a gender using this service with a different image, I always get the class "man" with the probability "0.008771929889917374".

I have tested with this list of images:

Here the command line I used to create the service:

curl -X PUT "http://localhost:8080/services/gender" -d "{\"mllib\":\"caffe\",\"description\":\"clothes classification\",\"type\":\"supervised\",\"parameters\":{\"input\":{\"connector\":\"image\",\"height\":224,\"width\":224},\"mllib\":{\"nclasses\":2}},\"model\":{\"repository\":\"/home/eurecom/Documents/deepdetect/gender\"}}"

Here the command line I used to predict the gender:

curl -X POST "http://localhost:8080/predict" -d "{\"service\":\"gender\",\"parameters\":{\"output\":{\"best\":5}},\"data\":[\"http://4.bp.blogspot.com/-uwu7SmTbBXI/VD_NNJc4Y-I/AAAAAAAAK1I/rt9de3mWXJo/s1600/faux-fur-coat-winter-2014-big-trend-10.jpg\"]}"

Did I do something wrong by creating the service? Did I do not use correctly the service? Or id this a bug?

Thanks in advance.

beniz commented 8 years ago

Thanks @jplu you are correct that the model is actually the wrong one. I've put what I believe to be the correct one online, can you try it ? I'm not in a position to immediately make the appropriate check and I figured you wouldn't mind trying the new tarball.

beniz commented 8 years ago

OK, I've been able to check & fix it (from a bus :)).

Please download the tar.bz2 file again and test.

Scores are actually perfect on your examples:

curl -X POST "http://localhost:8080/predict" -d "{\"service\":\"gender\",\"parameters\":{\"output\":{\"best\":114}},\"data\":[\"http://i.f1g.fr/media/ext/400x/madame.lefigaro.fr/sites/default/files/img/2015/08/les-mensurations-de-la-femme-ideale-a-travers-18-pays.jpg\"]}"
{"status":{"code":200,"msg":"OK"},"head":{"method":"/predict","time":4130.0,"service":"gender"},"body":{"predictions":{"uri":"http://i.f1g.fr/media/ext/400x/madame.lefigaro.fr/sites/default/files/img/2015/08/les-mensurations-de-la-femme-ideale-a-travers-18-pays.jpg","classes":[{"prob":1.0,"cat":"women"},{"prob":9.654188559693467e-10,"last":true,"cat":"men"}]}}}

and

curl -X POST "http://localhost:8080/predict" -d "{\"service\":\"gender\",\"parameters\":{\"output\":{\"best\":2}},\"data\":[\"http://www.bragard.fr/471/tridou-veste-homme-noir.jpg\"]}"
{"status":{"code":200,"msg":"OK"},"head":{"method":"/predict","time":6805.0,"service":"gender"},"body":{"predictions":{"uri":"http://www.bragard.fr/471/tridou-veste-homme-noir.jpg","classes":[{"prob":1.0,"cat":"men"},{"prob":8.472202495113379e-9,"last":true,"cat":"women"}]}}}

jplu commented 8 years ago

Thanks for you quick answer!

I just downloaded the the new version on the website and unfortunately the same issue still remain :(

With the three same pictures always "men" as category, and "0.008771929889917374" as probability.

beniz commented 8 years ago

Weird, check that there s a loss3/classifier_ftune layer at the end of the deploy.prototxt file in the model repository. Same applies to googlenet.prototxt.

jplu commented 8 years ago

This is the two last layers in deploy.prototxt:

layer {
  name: "loss3/classifier"
  type: "InnerProduct"
  bottom: "pool5/7x7_s1"
  top: "loss3/classifier"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  inner_product_param {
    num_output: 114
    weight_filler {
      type: "constant"
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}
layer {
  name: "prob"
  type: "Softmax"
  bottom: "loss3/classifier"
  top: "prob"
}

And here the two last layers in googlenet.prototxt:

layer {
  name: "loss3/top-1"
  type: "Accuracy"
  bottom: "loss3/classifier"
  bottom: "label"
  top: "loss3/top-1"
  include {
    phase: TEST
  }
}
layer {
  name: "probt"
  type: "Softmax"
  bottom: "loss3/classifier"
  top: "probt"
  include {
    phase: TEST
  }

beniz commented 8 years ago

Ah my bad moving the files over the network (from the bus!). Just re-download the tarball please, it should hopefully work.

jplu commented 8 years ago

It is working perfectly now. Thanks a lot!

jolibrain / deepdetect

Gender model always predict the same class with same probability #64