Coupled cross entropy does not accept ndArray or Tensor inputs

[Kevin-Chen0]

In the current vae code, x_true is the input images while x_recons_logits is the output generated images. Both are Tensors. In the tf lib, there is a cross entropy function that calculates these two:

        raw_cross_entropy = tf.nn.sigmoid_cross_entropy_with_logits(
        labels=x_true, logits=x_recons_logits)

However, in our nsc lib, our coupled_cross_entropy function inputs density function p and q. Would it also work if we input in x_true and x_recons_logits? Will it be something like:

coupled_cross_entropy(x, x_gen, sample_n)

Although how do we get sample_n?

[kenricnelson]

Yes, something like this is the idea; however, I think the issue which will have to be worked through is our coupled cross entropy assumes normalized distributions and also takes distribution parameters rather than points as inputs. The alternative is to just work with the coupled logarithm function. The weight multiplying the logarithm is x_true and the argument of the logarithm is x_reons_logits. Assuming x_true is black or white, 0 or 1, the equation is x_true coupled_log(1/x_recons_logits) + (1 - x_true) coupled_log(1/(1-x_recons_logits)). For non black & white images, I would assume that the code you showed either includes some kind of normalization so that pixel values represent probabilities or there is some justification why the normalization is not required.

[Kevin-Chen0]

The raw input image x can have a range of 0-255 on a grayscale spectrum, where 0 is black and 255 is white. But you can either normalize them to between 0-1 or make it binary (only 0 or 1). For the x_recons_logit you can either have the raw logit or after softmax function probs. However, raw logits from the NNs initially can include any real number including negatives.

In our current VAE model, the Input x is binary [0, 1] while the while x_recons_logit is any real numbers from the output decoder, with the expectations that the number converge to x to some degree. This VAE is derived from the code here, where you can see raw x is being preprocessed to be binary:

def preprocess_images(images):
  images = images.reshape((images.shape[0], 28, 28, 1)) / 255.
  return np.where(images > .5, 1.0, 0.0).astype('float32')

[kenricnelson]

Discussed this with John. Plan to make a function called coupled_cross_entropy_prob which will take probabilities rather than distribution parameters as an input. Also complete coupled_entropy_prob and coupled_kl_divergence_prob

this is high priority so that Coupled VAE experiments can begin

[Kevin-Chen0]

Great to hear. Please work with @hxyue1 on this as he will need to use this to integrate into the loss function of the VAE.

[jkclem]

We have a function that does this now. Closing issue.