google-deepmind / dsprites-dataset

Dataset to assess the disentanglement properties of unsupervised learning methods
Apache License 2.0
477 stars 69 forks source link

Regarding generative factors for the dataset #2

Closed prannayk closed 6 years ago

prannayk commented 6 years ago

How were the images created? What was the original size of the shapes (in pixels) and how did you transform them? (what algorithms specifically)

Basically what I am asking is that what are the true generative factors of the dataset?

Azhag commented 6 years ago

Heya, thanks for the interest!

I generated the images using Love2D, with some additional logic I wrote to handle the camera/transforms to place items around a bit more easily (came out of another project). Everything was vector-based, so they do not have a "original size" per say (i.e. I'm never loading a texture of fixed resolution). Instead, I really give whatever posX/posY you have in the dataset and use Love2D usual functions to draw vectors. Consider that the canvas has normalized size 1 in both width/height, that's usually how Love2D works.

Some of the shapes use direct drawing functions (e.g. square is https://love2d.org/wiki/love.graphics.rectangle, ellipse is https://love2d.org/wiki/love.graphics.ellipse), the heart I wrote custom functions to procedurally generate a list of vertices for Mesh which would then be rendered directly with the transform arguments appropriately set when drawing.

This is then rendered/rasterized by Love2D, at the 64x64 resolution you see the data as (no antialiasing or anything like that, defaults of setMode apart from setting width and height to 64). To avoid using generative factors values that would render the same pixel images (at 64x64 due to aliasing), I optimized the ranges and number of factors to avoid any duplicate among all images (hence why we don't have really small scales, or everything would just be tiny dots).

Feel free to check Love2D for the details of the rendering algorithms (but I would assume that the "transformations" are usual vector graphics, unclear what you want to know?), I used their https://love2d.org/wiki/love.graphics.translate, https://love2d.org/wiki/love.graphics.scale and https://love2d.org/wiki/love.graphics.rotate functions, sometimes combined as a single transformation matrix.

I would claim that the factors we provide are exactly the generative factors of the dataset, given that I explicitly fed them into the Love2D renderer and that they are sufficient statistics of any given image in the dataset. But maybe it's more of a philosophical discussion to have :)

prannayk commented 6 years ago

Thanks for the help. This is good. If it is possible, would you be okay sharing the code for generating the heart?

Azhag commented 6 years ago

I sadly don't have access to the exact original code right now (and it was in Lua), but I used the last formula on http://mathworld.wolfram.com/HeartCurve.html:

def heart(t):
  x = 16*np.sin(t)**3.
  y = 13*np.cos(t) - 5*np.cos(2*t) -2*np.cos(3*t) - np.cos(4*t)
  return x, y

With t between [0, 2pi], with enough points for the heart to look smooth (I would guess maybe 100 vertices from memory?)

t_space = np.linspace(0, 2*np.pi, 100)
heart_x, heart_y = heart(t_space)
plt.plot(heart_x, heart_y, 'r')
prannayk commented 6 years ago

Ah I see, the 2\theta curves. Thanks.