Open tportenier opened 4 years ago
Hi,
thank you for your interest and for spotting those bugs (Those things were some legacy lines that I must have missed when cleaning up the code)! Feel free to send a pull request!
Unfortunately, the noise sampler is a Cuda only implementation. I can give you some legacy code where I implemented the noise in python / pytorch only. This should run on cpu, but you might have to make some adjustments and permute / transpose the position and or seed tensors.
here is the code and some insights: outer_noise is the "noise sampler" that calls "middle_noise" and "inner_noise". Inner noise is the actual noise function and the other two are just helpers to sample bilinear noise and reshape tensor accordingly.
def outer_noise(coord, seed, octaves):
channel_noise_stack = []
for channel in range(3):
noise_stack = []
for octave in range(octaves):
resolution = 2 ** (octave + 1)
lattice_coord = coord[:, octave] * resolution
noise_octave = middle_noise(lattice_coord, seed[:, channel, octave])
noise_stack.append(noise_octave)
channel_noise_stack.append(torch.stack(noise_stack, dim=1))
return torch.stack(channel_noise_stack,dim=2)
def middle_noise(lattice_coord, seed):
coord_integer = torch.floor(lattice_coord)
bs = lattice_coord.shape[0]
n_dim = lattice_coord.shape[1]
list_nearest_neighbours = []
list_bilinear_neighbours = []
sub_texel_coord = lattice_coord - torch.floor(lattice_coord)
weights = bilinear_weights(sub_texel_coord)
for i in range(2 ** n_dim):
string = format(i, '0{}b'.format(n_dim))
offset = torch.tensor([0 if c == '0' else 1 for c in string], dtype=torch.uint8, device='cuda')
offset_batch = offset.expand(bs, n_dim)
offset_batch = offset_batch.view(bs, n_dim, 1, 1)
weights_picked_list = []
for idx, value in enumerate(offset):
weights_picked_list.append(weights[:, idx, value.item()])
weights_picked = torch.stack(weights_picked_list, dim=1)
weights_prod = torch.prod(weights_picked, dim=1, keepdim=True)
list_nearest_neighbours.append(inner_noise(coord_integer + offset_batch, seed))
list_bilinear_neighbours.append(inner_noise(coord_integer + offset_batch, seed) * weights_prod)
noise_nearest = torch.sum(torch.cat(list_nearest_neighbours, dim=1), dim=1, keepdim=True)
noise_bilinear = torch.sum(torch.cat(list_bilinear_neighbours, dim=1), dim=1, keepdim=True)
noise = torch.cat([noise_nearest, noise_bilinear], dim=1)
return noise
def inner_noise(integer_coord, seed):
bs, c, w, h = integer_coord.size()
PHI = 1.61803398874989484820459 * 00000.1
PI = 3.14159265358979323846264 * 00000.1
THETA = (3.14159265358979323846264 / 4.0) * 00000.1
SQ2 = 1.41421356237309504880169 * 10000.0
a = torch.tensor([PHI, PI, THETA])
b = torch.tensor(SQ2)
a = a.expand(bs, 3).unsqueeze(-1).cuda()
integer_coord = integer_coord.view(bs, c, -1)
integer_coord = integer_coord * (seed + PHI)
distance = torch.norm(integer_coord - a[:, :c], dim=1, keepdim=True)
noise = torch.tan(distance) * b
noise = noise - torch.floor(noise)
noise = noise.view(bs, 1, w, h)
return noise
First of all, this is great work, amazing!
While playing around with your code I encountered an issue relating to the synthesis of interpolated samples.
First of all,
s_neural_texture.py line 254: z_texture_interpolated = z_texture_interpolated[:, :-2]
does not work, since it causes a dimension mismatch later on. I don't know that it is supposed to do so I commented it out.Next,
s_neural_texture.py line 68+69
only make sense in non-interpolation mode, I therefore prepended the lineif z_encoding.shape[2]==1:
to mitigate this.Finally,
transform_coord()
inneural_texture_helper.py
is also not handling interpolation mode properly. I replaced the lines 105-107 with the following:An unrelated question: is it possible to somehow run interference on cpu? I am a tensorflow guy and not familiar with pytorch, but it seems that your custom noise sampler is not cpu capable. Is there a way to run it on the cpu?
Cheers, tiziano