microsoft / VoTT

Visual Object Tagging Tool: An electron app for building end to end Object Detection Models from Images and Videos.
MIT License
4.28k stars 833 forks source link

Labels are incorrect in Active Learning #812

Open dino-su opened 5 years ago

dino-su commented 5 years ago

Incorrect labels in Active Learning

2

The problem seems caused by the label offset in src/providers/activeLearning/objectDetection.ts and it can be fixed by apply following patch.

Patch file

diff --git a/src/providers/activeLearning/objectDetection.ts b/src/providers/activeLearning/objectDetection.ts
index 196db45..daf7edb 100755
--- a/src/providers/activeLearning/objectDetection.ts
+++ b/src/providers/activeLearning/objectDetection.ts
@@ -222,7 +222,7 @@ export class ObjectDetection {

     private getClass(index: number, indexes: Float32Array, classes: number[]): string {
         if (this.jsonClasses && index < indexes.length && indexes[index] < classes.length) {
-            const classId = classes[indexes[index]] - 1;
+            const classId = classes[indexes[index]];
             const classObject = this.jsonClasses[classId];

             return classObject ? classObject.displayName : strings.tags.warnings.unknownTagName;

After patch 3

matthieu-lapeyre commented 5 years ago

Same problem here, release v2.1.0 on MacOS

Capture d’écran 2019-05-29 à 17 34 47
csaroff commented 5 years ago

I submitted a PR last week to fix this issue. The update broke the tests which I haven't gotten around to fixing yet, but if you git clone https://github.com/csaroff/VoTT.git and then npm start, you should be good to go!

@dino-su It doesn't look like your patch fully resolves this issue. If you check out the classes.json file of the cocossd model, you can see that there are missing classes. This is the expected format of the coco classes. You can check my extremely brief PR for the details of the fix.

Dahlasam commented 4 years ago

The issue was still lurking as I cloned the git just today. I was able to get the labels right by changing the src/providers/activeLearning/objectDetection.ts:225 line to const classId = classes[indexes[index]]; // - 1; --> So as the labels were off by one in the index, so I removed the -1 and got this working. Maybe this brakes the generally correct functioning, so this is not a general fully tested fix by any means. The same 'one off label' issue was still reproducible using the Pre-trained COCO as well.