Increase training data on VAE

[Templar129]

VAE turns to be a quite good model when we want to generate similar yet new images. However, as our data has several patterns that is hard to capture, such as the geographical features, the delicate change in depth and the specific shape of our glacier, it would be much helpful if we can use multiple input training data instead of only one. There are two potential options. One is that we slice the original TG into many slices like a cake, and use the slices as input data. But the pieces of images are connected and related to each other, and the smaller pieces would create difficulties when we want to put generated data back to a larger picture. Another option would be using data augmentation. It is a algorithm that can generate different versions of original data, it keeps everything except it would rotate, or reflect the data, sometimes we can manually add some noises. This could be a better option because it preserves the whole picture and we would not lose too many features. I am now working a way to generate maybe 1000 new training data based on the original TG data and then put them into a VAE.

I think I can finish the script by Friday and I will upload a new folder to the fork and then keep updating it and documenting it.

[Templar129]

Now we have a script that can use a single image and create numerous different versions of it, and this way we can have a relatively large size training data set, which is critical to training of VAE. Now a training dataset containing 500 images takes 6 Gibs and more, and it takes a lot of RAM on local machine. Also, for a VAE to have better performance, we might considering adding more layers, which also is a computation consuming task. Thus I think we should move the VAE model to our JupyterHub, since it provides us with high computational availabilities and well-configured tensorflow env. I have just uploaded my data augmentation scripts to the Hub and will later transfer the previous VAE model there. By the limit of my current laptop, the previous model was a quite really simple one, with a few layers and does not perform well at all. I think on the Hub we can have a much better model. I will work on it now and a developed model might take one day or so.

[Templar129]

Jul 28 updates: After a few configuring on the JupyterHub, now, it is okay to run several VAE models. I have been trying to build a VAE models on the Hub since our data has quite large size it has (904 x 1744) pixels per image. Until now I have tried: GAN models with 32 x 32 input, I have split the original image into smaller pieces and put it into the GAN, it can produce several images but with a lot of white noise spots. VAE with original size and 20,50,99 images each time, they all crashed due to lack of RAM. Even the smallest subset of data, 20 images would cause OOM(out of memory). I believe this is because of lack of efficiency of my codes, so I am still learning and trying to develop it, but it would definitely take some time. VAE with 32x32 size. This size is really easy for the VAE, so it overfitted. It does not give any meaningful output, simply mosaic black and white images which could not help us.

Because of the complexity of GAN models, it works well with input size smaller than 64 x 64, but it provides high resolution and sharpness, which means if we decide to use smaller pieces of image, GAN would work better than the VAE. VAE takes less RAM than GAN but it works when the image has higher resolution but this demand on the other hand takes a lot of RAMs.

I think right now I would keep working on developing the VAE, try to optimize the code so it would not crash.

By the way, I think there might be another way of generating topologies. Sometimes video games makers would need to produce a lot of territories, and they usually do not create those mountains and lands by hand, they have a method called fractal generation. I will look this up when I have spare time.

[Templar129]

Hi everyone, I have just finished configuring the new VAE model. I have tried two different ways, one is to feed it with 20 identical TG plots and the result is that it can perfectly produce copies of the original data which is not what we wanted for a VAE. The other way I used is to feed it with splits of the original TG data. I split the TG plot into 20 pieces with same size and give it to the VAE, then it can produce several fusion plots. We can still observe some of the features we want from the latent spcae:

The next step would be do some hyper-parameter configuring. In the mean time, maybe we should feed the model with more diversed input data.

[Templar129]

Based on our discussion Andrew, we think there could be two potential way to overcome the disruption caused by the white background. First method is to crop the original data again and keep only the central part which means we give up the data on the edge but in exchange get multiple full size data. Method two is to using more new data as input. Andrew also mentions another data source that we can use for the model, it could provide us much more data so we can improve the quality of this model by adding much more training data. Currently I have been working on the first method. Cropping is not quite hard, but the cropped data is not quite compatible with the original model, I have been doing configuring on Wednesday, and I think the model would be good to go by today.

[Templar129]

Update on Aug 11:

Using method 1, I have generated 144 smaller images. Each one of them has 128 pixels x 128 pixels. Using these smaller images and the updated codes, I now have the new VAE model. It can now generate new and similar images to the input.

However there are still two problems with this model. One is that the model still suffers from high noise, which people can observe some white noise from generated data. The other problem is that if we generate two new images and put them together, we can still see a obvious line between two images. This is because the two images are generated stochastically, and not continuous. This could be a problem for later stages, but for now, I will focus on solving the first problem.

My initial thoughts are adding the complicity of the model which may use a lot more computation resources and take much more time. I will update here for the further improvement of the model.

Below is some of the images generated. I used 001m topology of both upstream and downstream TG data as input.

[Templar129]

I have just uploaded the VAE model to my fork. It uses upstream and downstream 001m topology as input data. I also included 5 demos that produced by the model. I will keep working on this model, and make another update once it has a major improvement. https://github.com/Templar129/Autoencoder_demo.git

[Templar129]

Update: The newest model now uses much more data as training data. It contains Russel, TG, and Humboldt. Each individual image has 512-pixel x 512-pixel resolution and there are 38 images in total. The new model also increased the number of convolutional layers, and latent dimensions. Here is the link to the fork: https://github.com/Templar129/Autoencoder_demo.git The new codes is stored in VAE_on_hub and named by 512x512. I also included a few new images it generated.

We can still observe a few problems about the model. First of all, because there are a lot of blank space in the original images, the model somehow learned this pattern and also tend to produce this same blank spaces. Another problem would be that once we have got a few desirable results, we would have to put them together to make a whole, big plot. But the images are not continuous, so they must have sharp edges between each other.

[Templar129]

Here is a simple demo image generated: