cameras_xxx.npz

[smoreira00]

What is the meaning of each element of 'cameras_xxx.npz', i.e., of 'camera_mat_xx', 'camera_mat_inv_xx', 'world_mat_xx', 'world_mat_inv_xx', and 'scale_mat_xx'?

[olkovi]

The world matrix is made by multiplying the extrinsic matrix (E) by the intrinsic matrix (I).

The extrinsic matrix stores the transformation needed to go from the world reference frame to the camera reference frame, by describing the camera's rotation and translation.

After that is done, the intrinsic matrix to points to a specific pixel on the camera's film using its properties, i.e. the sensor size, the lens focal length over x and over y, the number of pixels in x and in y directions.

Instead of performing two matrix multiplications at runtime, W = I x E is done prior, which saves time but deprives you of non-entangled camera translation parameters. To get them back you will need to make your own intrinsic matrix, provided you do know the camera parameters.

The inverse gets you from a pixel to a point in the world space. I believe the camera matrix is another name for the world matrix, but it might mean its inverse, depending on the author. Check this yourself.

Here is now the matrices go together:

An illustration of the transformation:

The scale matrix increases or decreases the world size with respect to NeuS's unit sphere. E.g. if your scene isn't fitting inside the sphere, increase the scale matrix diagonal from 1 to 4, 16 etc. until it does.

[smoreira00]

How do I compute the scale matrix so the scene fits inside that sphere? Could you give me an example? Is there any code to obtain the scale matrices?

[olkovi]

Well, if you know the size of your scene, then the sphere radius should allow you to encompass it. The authors generally use scenes less than 2 metres in size, so they just use scale = 1. If you wanted to automate that, you would need to read the maximum distance between two points in your mesh and set the scale/2 to just above that by maybe 5% or so.

Until you write that code, just set the scale to what it needs to be, like here:

import numpy as np
experiment_name = input('enter folder name: ') +'/'
a = dict(np.load(experiment_name+'cameras_sphere.npz'))
print('base scale == 4')
print('current scale is ' + str(a['scale_mat_0'][0][0]))
c = int(input('enter new scale as integer: '))
new_scale = np.array(([c,0,0,0],
                         [0,c,0,0],
                         [0,0,c,0],
                         [0,0,0,1]),dtype = float)
scale_inv = np.linalg.inv(new_scale)
ns = 0
for i in a:
    print(i)
    if ns % 4 == 0:
        a[i] = new_scale
        print(a[i])
    elif ns % 4 == 1:
        a[i] = scale_inv
        print(a[i])
    ns+=1
np.savez(experiment_name+"cameras_sphere.npz", **a) #np.save save as .npy