Replicating SCAN algorithm described in Google DeepMind's paper "SCAN: Learning Abstract Hierarchical Compositional Visual Concepts"
Image datasets are created with Rodent envrironment.
First extract dataset, and then run main.py
$ tar xvf data.tar.gz
$ python main.py
Symbol to image conversion result.
1) Generated images when wall_color=white
is specified.
2) Generated images when wall_color=white
, floor_color=white
are specified.
3) Generated images when wall_color=white
, floor_color=white
, obj_color=white
are specified.
4) Generated images when wall_color=white
, floor_color=white
, obj_color=white
, obj_id=ice_lolly
are specified.
Input | Output |
---|---|
obj_color=white , wall_color=white , floor_color=white , obj_id=ice_lolly |
(All of the outputs are correct.)
Input | Output |
---|---|
obj_color=purple , wall_color=dark_yellow , obj_id=hat |
(Correct obj_color
was red
, but confused as purple
. floor_color
was not specifiled in the output.)
Concept reombination result as image outputs.
1) Recombination result of (obj_color=white) AND (wall_color=white)
. (Correct result should have white object and white wall.)
2) Recombination result of (obj_color=white, obj_id=ice_lolly) IN_COMMON (obj_color=white, wall_color=white
. (Correct result should have white object.)
3) Recombination result of (obj_color=white, wall_color=white) IGNORE (obj_color=white)
. (Correct result should have white wall.)
Disentanglement result for latent variables for object parameters. (Wall color, Object color, Floor Color, Object Type, Object position).
Wall color
Obj color
Floor color
Obj Type
Obj pos (and Obj Type)