Open aksg87 opened 2 years ago
Hi! Answers to your questions:
calc_ssim_from_checkpoint
-> I simply had not added SSIM as a metric to tensorboard yet when I wrote this script, so this file can be ignored (or removed) now.decode_embeddings.py
-> the db_path
are the generated embeddings by your autoregressive model, so you don't have them right now.extract_embeddings.py
-> yes, this file in principle takes your model + dataset and created the embeddings which should be used as training input for your autoregressive model.Nice to see that you're progressing :)
@robogast - Appreciate all of the information! Need to review the paper again :)
I look forward to trying the other scripts and posting how things go!
Hi @robogast
Your comments make much more sense now after reviewing the literature further :)
This is a nice overview from AI Epiphany!
Hi @robogast
I was trying to better understand encoding_idx
. My understanding is that this is the last item in each of the 3 bottle neck layers? Curious why we throw the rest of the information away?
Thanks in advance! -Akshay
def extract_samples(model, dataloader):
model.eval()
model.to(GPU)
with torch.no_grad():
for sample, _ in dataloader:
sample = sample.to(GPU)
*_, encoding_idx = zip(*model.encode(sample))
yield encoding_idx
@robogast
Happy to report I was able to train a VQ-VAE using a dataset. Very cool to see - and kudos for the nice Tensorboard outputs you have in place! 😎
Do you have any suggestions or code for randomly sampling from the decoder in a generative fashion?
Also, If you have a summary of these files and their purpose, that would be very helpful. I would be happy to do a PR with some comments in the repository if that would be helpful.
Questions on: calc_ssim_from_checkpoint.py # does this calculate SSIM across the dataset ❓ decode_embeddings.py # Specifications for db_path ❓ extract_embeddings.py # Does this save embedding to disk ❓
Ran successfully: plot_from_checkpoint.py # plots a forward pass from a random sample ✅ train.py # trains a model ✅
Much appreciated! -Akshay