search

LucaCappelletti94 / extra_keras_metrics

Additional training metrics integrated with the keras NN library.
MIT License
13 stars 2 forks source link

Iou metric #3

Open sainatarajan opened 5 years ago

sainatarajan commented 5 years ago

Hi, thanks for the library. Does it contain the mean_iou metric?

LucaCappelletti94 commented 5 years ago

It is not currently implemented, let's see what I can do about it.

sainatarajan commented 5 years ago

Thank you.

LucaCappelletti94 commented 5 years ago

I have ported mean_iou from Tensorflow into the library. I have tested that it runs, but I have no comparison metric from sklearn or other libraries to test it against. Do let me know if it behaves properly.

P.S. If you need to implement another metric you can also do a pull request, the way I'm porting the metrics is really easy since I've made a couple of decorators that make it immediate. P.P.S. If you like the repo consider starring it! :)

sainatarajan commented 5 years ago

Thanks. That was really quick. I will let you know about its behavior after comparing it with other libraries.

LucaCappelletti94 commented 5 years ago

I will close this issue now, let me know if anything weird happens.

sainatarajan commented 4 years ago

Thanks. I was able to test only now. It works as expected. But I was not able to use the mean_iou in the metrics parameter of model.compile() method. It says Unknown metric function: mean_iou.

sainatarajan commented 4 years ago

I just realized that it is a parameterized metric. So, I instantiated it with 1 i.e. mean_iou(1), and got the following error.

InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument: assertion failed: [`labels` out of bound] [Condition x < y did not hold element-wise:] [x (metrics_2/mean_iou1/mean_iou/confusion_matrix/control_dependency:0) = ] [1 0 0...] [y (metrics_2/mean_iou1/mean_iou/Cast:0) = ] [1]
     [[{{node metrics_2/mean_iou1/mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert}}]]
     [[ConstantFoldingCtrl/metrics_2/mean_iou1/mean_iou/confusion_matrix/assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Switch_0/_3294]]
  (1) Invalid argument: assertion failed: [`labels` out of bound] [Condition x < y did not hold element-wise:] [x (metrics_2/mean_iou1/mean_iou/confusion_matrix/control_dependency:0) = ] [1 0 0...] [y (metrics_2/mean_iou1/mean_iou/Cast:0) = ] [1]
     [[{{node metrics_2/mean_iou1/mean_iou/confusion_matrix/assert_less/Assert/AssertGuard/Assert}}]]
0 successful operations.
0 derived errors ignored.
LucaCappelletti94 commented 4 years ago

Could you please add a minimal code snippet to reproduce the error?

sainatarajan commented 4 years ago

Hi, here is the snippet. model.compile(optimizer= 'adam', loss= total_loss, metrics= ['root_mean_squared_error', mean_iou(5)]) Error is same as before.

LucaCappelletti94 commented 4 years ago

Could you please also add the model etc..., so that we can use your snippet as test? Thanks!

sainatarajan commented 4 years ago

Sure. Here it is. I am trying to run the deeplabv3 model from this repo. Here is the small code:

from extra_keras_metrics import mean_iou
# compile Keras model with the defined optimizer, loss, and metrics
deeplab_model.compile(optimizer= 'adam', loss= 'categorical_crossentropy', metrics= ['root_mean_squared_error', mean_iou(5)])

Also, when I add more metrics, the process stops execution and reports ResourceExhaustedError. I think this one is not due to this library. If you need the full code, I will send it to you in email.

LucaCappelletti94 commented 4 years ago

Ok, but I need a small snippet of code to fully reproduce the issue, I don't have the time currently to read the repository.

Please provide a complete snippet to reproduce the issue, otherwise it would take too long.

Thanks!