rosinusserrano / pml_vqvae

Repository for the course "Project Machine Learning" during WiSe 24/25 at TU Berlin consisting of a replication of the paper "Neural Discrete Representation Learning" (van den Oord et al., 2018).
0 stars 0 forks source link

Use tanh / sigmoid activation in last layer #15

Open rosinusserrano opened 3 days ago

rosinusserrano commented 3 days ago

Since our model outputs images in range [0, 1] it makes a lot of sense to use sigmoid activation in the last layer which maps the values into that exact range. Furthermore we could think of mapping our images into a [-1, 1] range and then using tanh activation which is thought to be better than sigmoid: https://datascience.stackexchange.com/questions/54296/should-input-images-be-normalized-to-1-to-1-or-0-to-1