Prepare a demo/test DB of about 10-15 (holiday?) images

[ridderl]

When looking for demo pictures, I came across: http://www.forensicgenealogy.info/contest.html (scroll down to see images, with questions). Not sure we can use it, maybe just for inspiration :-)

[ridderl]

How about testing our car model on pictures of crashed cars and present the results as a demo. It is both a test of the robustness of the model and has forensic relevance also... https://www.google.nl/search?q=foto+auto+onderzoek&biw=1920&bih=919&espv=2&tbm=isch&tbo=u&source=univ&sa=X&ved=0ahUKEwjpzp6t0NHMAhWCOSwKHRi1Cs8QsAQIGw&dpr=1#tbm=isch&q=auto+ongeluk

Let me know what you think ...

[ridderl]

Oh and thirdly: those crashed car images often have multiple car (remainders) on one picture, so we can test the bounding box search as well. Maybe it is actually a bit too challenging altogether :-)

[elboyran]

I propose to use the COCO dataset for test images. Given the VM folder access I can put those for now in /data/elena_data. They should be better in /data/demo_data. Made a new issue #108 for that.

[elboyran]

24 images from COCO + 1 car image are now in /data/elena_data.

[elboyran]

The current test images are too challenging! :-( Here's what our pipeline gives (quite crap 👎 ).

In file '/data/cyclist_dog.jpg' top object 'skis' has been detected! In file '/data/cows.jpg' top object 'cow' has been detected! In file '/data/cow_street.jpg' top object 'potted plant' has been detected! In file '/data/kitten_puppy.jpg' top object 'bear' has been detected! In file '/data/car_bikes_boats_canal.jpg' top object 'chair' has been detected! In file '/data/truck.jpg' top object 'truck' has been detected! In file '/data/bottles_glas.jpg' top object 'dining table' has been detected! In file '/data/girl_two_dogs.jpg' top object 'book' has been detected! In file '/data/motorbikes_riders_cars.jpg' top object 'cup' has been detected! In file '/data/street_car_toyota.jpg' top object 'clock' has been detected! In file '/data/car_skateboard_man.jpg' top object 'chair' has been detected! In file '/data/screen_laptop_keyboard.jpg' top object 'cell phone' has been detected! In file '/data/person_horse.jpg' top object 'umbrella' has been detected! In file '/data/car_splash_small_person.jpg' top object 'boat' has been detected! In file '/data/oranges_apples_bananas.jpg' top object 'dining table' has been detected! In file '/data/bull.jpg' top object 'cow' has been detected! In file '/data/cirl_car_sofa.jpg' top object 'teddy bear' has been detected! In file '/data/young_man_birthday.jpg' top object 'refrigerator' has been detected! In file '/data/car_bus_old.jpg' top object 'umbrella' has been detected! In file '/data/Fiat_panda_auto_zijkant.jpg' top object 'car' has been detected! In file '/data/fire_hydrant_car.jpg' top object 'fire hydrant' has been detected! In file '/data/kitten_dog.jpg' top object 'couch' has been detected! In file '/data/old_man_baby_mobile.jpg' top object 'cell phone' has been detected! In file '/data/two_riders_horses.jpg' top object 'horse' has been detected! In file '/data/young_man_woman_wedding.jpg' top object 'dining table' has been detected!

We need easier images for a demo IMHO. Please, whoever is free to pick this issue up. The test images are in /data/demo_data on the VM.

[ridderl]

Are these the results obtained with the Yolo detector, or is it still the imagenet result?

[elboyran]

You are confusing 2 things, I think, imagenet classification is the finer one coming after the initial object detector. I.e. the choices at first level are Yolo/SSD, later imagenet/car color/car model/face/ age/gender....

Berend knows for sure, but I think this has been obtained by the SSD.

What i gave are only the top level detections (after step 1- obj detector). First step only in this image

pipeline schema

[ridderl]

Sorry, then I meant yolo vs SSD. My earlier test with yolo was quite OK, so that is why I asked.

[elboyran]

Berend said he made now Yolo work, but he gave me this result yesterday with SSD. Still, not sure if we have hope with Yolo as well. Please, verify with him.

[ridderl]

Looked at the detect.json after yolo detection and it seems fine at first sight.

[elboyran]

Hm, indeed, Yolo's output is now much better! Maybe we don't need new images? Lets see how they look after visualization...

In file 'girl_two_dogs.jpg' top object 'dog' has been detected! In file 'girl_two_dogs.jpg' top object 'dog' has been detected! In file 'girl_two_dogs.jpg' top object 'person' has been detected! In file 'two_riders_horses.jpg' top object 'horse' has been detected! In file 'two_riders_horses.jpg' top object 'horse' has been detected! In file 'two_riders_horses.jpg' top object 'person' has been detected! In file 'two_riders_horses.jpg' top object 'person' has been detected! In file 'old_man_baby_mobile.jpg' top object 'cell phone' has been detected! In file 'old_man_baby_mobile.jpg' top object 'person' has been detected! In file 'old_man_baby_mobile.jpg' top object 'person' has been detected! In file 'car_splash_small_person.jpg' top object 'boat' has been detected! In file 'car_splash_small_person.jpg' top object 'boat' has been detected! In file 'car_splash_small_person.jpg' top object 'car' has been detected! In file 'car_splash_small_person.jpg' top object 'car' has been detected! In file 'car_splash_small_person.jpg' top object 'person' has been detected! In file 'car_bus_old.jpg' top object 'bus' has been detected! In file 'car_bus_old.jpg' top object 'car' has been detected! In file 'car_bus_old.jpg' top object 'person' has been detected! In file 'car_bus_old.jpg' top object 'person' has been detected! In file 'car_bus_old.jpg' top object 'truck' has been detected! In file 'Fiat_panda_auto_zijkant.jpg' top object 'car' has been detected! In file 'truck.jpg' top object 'car' has been detected! In file 'truck.jpg' top object 'truck' has been detected! In file 'person_horse.jpg' top object 'horse' has been detected! In file 'person_horse.jpg' top object 'person' has been detected! In file 'cows.jpg' top object 'cow' has been detected! In file 'cows.jpg' top object 'cow' has been detected! In file 'cows.jpg' top object 'cow' has been detected! In file 'cows.jpg' top object 'cow' has been detected! In file 'fire_hydrant_car.jpg' top object 'fire hydrant' has been detected! In file 'fire_hydrant_car.jpg' top object 'fire hydrant' has been detected! In file 'fire_hydrant_car.jpg' top object 'car' has been detected! In file 'fire_hydrant_car.jpg' top object 'car' has been detected! In file 'screen_laptop_keyboard.jpg' top object 'laptop' has been detected! In file 'screen_laptop_keyboard.jpg' top object 'keyboard' has been detected! In file 'screen_laptop_keyboard.jpg' top object 'tvmonitor' has been detected! In file 'screen_laptop_keyboard.jpg' top object 'mouse' has been detected! In file 'screen_laptop_keyboard.jpg' top object 'cell phone' has been detected! In file 'kitten_puppy.jpg' top object 'dog' has been detected! In file 'kitten_puppy.jpg' top object 'bird' has been detected! In file 'kitten_dog.jpg' top object 'dog' has been detected! In file 'kitten_dog.jpg' top object 'cat' has been detected! In file 'bottles_glas.jpg' top object 'chair' has been detected! In file 'bottles_glas.jpg' top object 'bottle' has been detected! In file 'bottles_glas.jpg' top object 'bottle' has been detected! In file 'bottles_glas.jpg' top object 'wine glass' has been detected! In file 'young_man_birthday.jpg' top object 'cup' has been detected! In file 'young_man_birthday.jpg' top object 'bowl' has been detected! In file 'young_man_birthday.jpg' top object 'bowl' has been detected! In file 'young_man_birthday.jpg' top object 'bowl' has been detected! In file 'young_man_birthday.jpg' top object 'knife' has been detected! In file 'young_man_birthday.jpg' top object 'diningtable' has been detected! In file 'young_man_birthday.jpg' top object 'person' has been detected! In file 'young_man_birthday.jpg' top object 'cake' has been detected! In file 'cyclist_dog.jpg' top object 'bicycle' has been detected! In file 'cyclist_dog.jpg' top object 'dog' has been detected! In file 'cyclist_dog.jpg' top object 'person' has been detected! In file 'bull.jpg' top object 'cow' has been detected! In file 'young_man_woman_wedding.jpg' top object 'diningtable' has been detected! In file 'young_man_woman_wedding.jpg' top object 'person' has been detected! In file 'young_man_woman_wedding.jpg' top object 'person' has been detected! In file 'young_man_woman_wedding.jpg' top object 'cake' has been detected! In file 'young_man_woman_wedding.jpg' top object 'cake' has been detected! In file 'oranges_apples_bananas.jpg' top object 'apple' has been detected! In file 'oranges_apples_bananas.jpg' top object 'bowl' has been detected! In file 'oranges_apples_bananas.jpg' top object 'orange' has been detected! In file 'oranges_apples_bananas.jpg' top object 'orange' has been detected! In file 'oranges_apples_bananas.jpg' top object 'orange' has been detected! In file 'oranges_apples_bananas.jpg' top object 'banana' has been detected! In file 'cow_street.jpg' top object 'cow' has been detected! In file 'cirl_car_sofa.jpg' top object 'chair' has been detected! In file 'cirl_car_sofa.jpg' top object 'person' has been detected! In file 'cirl_car_sofa.jpg' top object 'sofa' has been detected! In file 'car_bikes_boats_canal.jpg' top object 'bicycle' has been detected! In file 'car_bikes_boats_canal.jpg' top object 'bicycle' has been detected! In file 'car_bikes_boats_canal.jpg' top object 'bicycle' has been detected! In file 'car_bikes_boats_canal.jpg' top object 'bicycle' has been detected! In file 'car_bikes_boats_canal.jpg' top object 'boat' has been detected! In file 'car_bikes_boats_canal.jpg' top object 'boat' has been detected! In file 'car_bikes_boats_canal.jpg' top object 'car' has been detected! In file 'motorbikes_riders_cars.jpg' top object 'fire hydrant' has been detected! In file 'motorbikes_riders_cars.jpg' top object 'motorbike' has been detected! In file 'motorbikes_riders_cars.jpg' top object 'motorbike' has been detected! In file 'motorbikes_riders_cars.jpg' top object 'car' has been detected! In file 'motorbikes_riders_cars.jpg' top object 'car' has been detected! In file 'motorbikes_riders_cars.jpg' top object 'person' has been detected! In file 'car_skateboard_man.jpg' top object 'car' has been detected! In file 'car_skateboard_man.jpg' top object 'car' has been detected! In file 'car_skateboard_man.jpg' top object 'car' has been detected! In file 'car_skateboard_man.jpg' top object 'car' has been detected! In file 'car_skateboard_man.jpg' top object 'skateboard' has been detected! In file 'car_skateboard_man.jpg' top object 'person' has been detected! In file 'street_car_toyota.jpg' top object 'motorbike' has been detected! In file 'street_car_toyota.jpg' top object 'car' has been detected! In file 'street_car_toyota.jpg' top object 'car' has been detected! In file 'street_car_toyota.jpg' top object 'car' has been detected! In file 'street_car_toyota.jpg' top object 'car' has been detected! In file 'street_car_toyota.jpg' top object 'car' has been detected!

[ridderl]

Is there something more I can do: how about a script that automatically generates a summary report? Listing the number of images per object class, a histogram of the number of objects per image, that sort of thing. Also I was thinking of a command line tool that helps querying the data.

[elboyran]

Looks like a good idea to me!

[elboyran]

I think we need a new issue for that. Please have a look a the 'images' section from the appended JSON file That can be used to generate the report per image(?)

[ridderl]

Yes it could.

[elboyran]

Have in mind that could still change, Maarten now complains it's not useful still for visualization. We are currently discussing it...

[ridderl]

That is OK. I will not use that section, but read directly from the other part.

[elboyran]

Well, reading from the other part is more painful and your code will have to change....

[ridderl]

I pushed a first version of a command line script to query the result. scripts/query.py

python scripts/query.py -h
usage: query.py [-h] json_filename {classes,images,class,image} ...

positional arguments:
  json_filename         JSON file

optional arguments:
  -h, --help            show this help message and exit

Sub-commands:
  {classes,images,class,image}
    classes             Return the list of detected classes
    images              Return the list of scanned images
    class               Return the images containing a classes
    image               Return the list of classes detected in an image

[elboyran]

I am afraid also the current pipleine JSON has changed.... but there's no example yet.

[ridderl]

$ python scripts/query.py JSON/example-output/output_demo.json  images
girl_two_dogs.jpg               3
two_riders_horses.jpg           4
old_man_baby_mobile.jpg         3
car_splash_small_person.jpg     5
car_bus_old.jpg                 5
Fiat_panda_auto_zijkant.jpg     1
truck.jpg                       2
person_horse.jpg                2
cows.jpg                        4
fire_hydrant_car.jpg            4
screen_laptop_keyboard.jpg      5
kitten_puppy.jpg                2
kitten_dog.jpg                  2
bottles_glas.jpg                4
young_man_birthday.jpg          8
cyclist_dog.jpg                 3
bull.jpg                        1
young_man_woman_wedding.jpg     5
oranges_apples_bananas.jpg      6
cow_street.jpg                  1
cirl_car_sofa.jpg               3
car_bikes_boats_canal.jpg       7
motorbikes_riders_cars.jpg      6
car_skateboard_man.jpg          6
street_car_toyota.jpg           6

$ python scripts/query.py JSON/example-output/output_demo.json  image car_bus_old.jpg
bus
car
person
person
truck

[ridderl]

No problem if the json changes, I can adapt it. My question is: is this type of script useful for the demo? I can further extend the options with new features.

[elboyran]

Ask the team leader ;-)

[ridderl]

query.py now also gives additional results from classifiers if available, e.g.:

$ python scripts/query.py JSON/example-output/output_demo.json image motorbikes_riders_cars.jpg
Classes in image: motorbikes_riders_cars.jpg Probability
fire hydrant                            0.2600
motorbike                               0.6700
motorbike                               0.6300
car                                     0.4700
     green                              0.9093
     Ssangyong Rodius                   0.0014
     Citroen C1                         0.0013
     Renault Grand Espace               0.0014
     Hyundai Atos                       0.0014
     Citroen C3 Picasso                 0.0013
car                                     0.7100
     Opel Omega                         0.0016
     Suzuki Baleno                      0.0013
     Citroen Xantia                     0.0013
     Subaru Justy                       0.0014
     Opel Vectra                        0.0013
person                                  0.6400

For the classification based on cropped faces it doesn't work yet as it currently goes only "one level deep"

[cwmeijer]

This script looks very nice! Definitely a nice demo I would say.