difference of probability computation per point between get_3DmFV in utils.py and get_3dmfv in tf_util.py

[JR9124]

thinks for sharing you work In utils.py and tf_util.py, you have realized 3DmFV using numpy and tensorflow. However, I find there is a little different between them. In utils.py(line 292 to 294), you directly assigned:
w_p = p_per_point Q = w_p # enforcing the assumption that the sum is 1 Q_per_d = np.tile(np.expand_dims(Q, -1), [1, 1, 1, D]) while in tf_util.py(line 610 to 613), you did this: w_p = tf.multiply(p_per_point,batch_w) Q = w_p/tf.tile(tf.expand_dims(tf.reduce_sum(w_p, axis=-1), -1),[1, 1, n_gaussians]) Q_per_d = tf.tile(tf.expand_dims(Q, -1), [1, 1, 1, D]) I think the two operations can achieve the same result as elements in batch_w are all the same value and both numerator and denominator of the following equation have the same w which can be eliminated, thus to enforce the assumption(line 610 and 612 in tf_util.py) is not necessary. However, after I use the following code to test (test data is car_130.txt from your 3DmFV-Tutorial-master), I get different results (by the above analysis, I think Q_per_d, Q_per_d0 and Q_per_d1 should equal to each other). ` n_gaussians = 8 variance = np.square(1.0/n_gaussians) gmm = utils.get_grid_gmm(subdivisions=[n_gaussians, n_gaussians, n_gaussians], variance=variance) points = utils.load_point_cloud_from_txt('car_130.txt') points = np.expand_dims(points, axis=0) import tensorflow as tf n_batches = points.shape[0] n_points = points.shape[1] ngaussians = gmm.means.shape[0] D = gmm.means_.shape[1]

# Expand dimension for batch compatibility
batch_sig = np.tile(np.expand_dims(gmm.covariances_, 0), [n_points, 1, 1])  # n_points X n_gaussians X D
batch_sig = np.tile(np.expand_dims(batch_sig, 0), [n_batches, 1, 1, 1])  # n_batches X n_points X n_gaussians X D
batch_mu = np.tile(np.expand_dims(gmm.means_, 0), [n_points, 1, 1])  # n_points X n_gaussians X D
batch_mu = np.tile(np.expand_dims(batch_mu, 0), [n_batches, 1, 1, 1])  # n_batches X n_points X n_gaussians X D
batch_w = np.tile(np.expand_dims(np.expand_dims(gmm.weights_, 0), 0), [n_batches, n_points,
                                                            1])  # n_batches X n_points X n_guassians X D  - should check what happens when weights change
batch_points = np.tile(np.expand_dims(points, -2), [1, 1, n_gaussians,
                                                    1])  # n_batchesXn_pointsXn_gaussians_D  # Generating the number of points for each gaussian for separate computation
p_per_point1 = (1.0 / (np.power(2.0 * np.pi, D / 2.0) * np.power(batch_sig[:, :, :, 0], D))) * np.exp(
    -0.5 * np.sum(np.square((batch_points - batch_mu) / batch_sig), axis=3))
Q_per_d1 = np.tile(np.expand_dims(p_per_point1, -1), [1, 1, 1, D])

mvn = tf.contrib.distributions.MultivariateNormalDiag(loc=batch_mu, scale_diag=batch_sig)
#Compute probability per point
p_per_point = mvn.prob(batch_points)# # n_batches X n_points X n_guassians*n_guassians*n_guassians
w_p = tf.multiply(p_per_point,batch_w)
Q = w_p/tf.tile(tf.expand_dims(tf.reduce_sum(w_p, axis=-1), -1),[1, 1, n_gaussians])
Q_per_d0 = tf.tile(tf.expand_dims(Q, -1), [1, 1, 1, D])
w_p = p_per_point
Q = w_p
Q_per_d = tf.tile(tf.expand_dims(Q, -1), [1, 1, 1, D])
with tf.Session() as sess:
    p_per_point = p_per_point.eval()
    Q_per_d = Q_per_d.eval()
    Q_per_d0 = Q_per_d0.eval()
difp = p_per_point - p_per_point1
difQ_per_d = Q_per_d - Q_per_d1
difQ_per_d0 = Q_per_d - Q_per_d0
print(sum(sum(np.squeeze(difp, axis=0))))# -1.4404886240351694e-10
print(sum(sum(np.squeeze(difQ_per_d, axis=0))))# [-81559.81403891 -81559.81403891 -81559.81403891]
print(sum(sum(np.squeeze(difQ_per_d0, axis=0))))# [-1.44048862e-10 -1.44048862e-10 -1.44048862e-10]

` I think get_3dmfv in tf_util.py can help to get right 3DmFV, but after analysis, I haven't found wrong place in get_3DmFV( in utils.py ), so could you tell me the reason? thank you!

[sitzikbs]

I see you closed this, did you find the problem ? If not, let me know and I will look into it.

[JR9124]

The sum of Q in get_3DmFV in utils.py is not equal to 1 while you enforce the sum of Q to be 1 in get_3dmfv in tf_util.py. That's the reason. BTW, do you think is it effective of reshaping fv to be [batch_size, 20, resresres] instead of [batch_size, 20, res, res, res] and training by deep neural network? Thanks!

[sitzikbs]

It is my experience that it will not be as effective since you will lose the prior knowledge of the spatial connections. Regarding the sum to one. They should be the same. I will look into it and try to push a fix soon. (the numpy version was not used in the paper and was created just for visualization purposes).

[JR9124]

Thanks for your advice. I have tested the code by changing w_p = p_per_point Q = w_p # enforcing the assumption that the sum is 1 to w_p = np.multiply(p_per_point, batch_w) Q = w_p / np.tile(np.expand_dims(np.sum(w_p, axis=-1), -1), [1, 1, n_gaussians]) then, the results obtained by get_3DmFV in utils.py and get_3dmfv in tf_util.py are the same.