Direct colors as vector field mismatch

[marcomusy]

VTK seems to be reinterpreting/resetting mapper.SetColorModeToDirectScalars() after a dataset transformation:

from vedo import *

pic = Image(dataurl+ "images/vtk_logo.png", channels=4)
msh = pic.tomesh()

arr = msh.pointdata["RGBA"].copy()
arr[arr[:,3] < 1] = [100, 50, 50, 200]

msh.pointdata["RGBA_TEST"] = arr[:, (0,1,2)]
msh.pointdata.select("RGBA_TEST") # DOES NOT WORK HERE

print(msh)

# import vtk
# tr = vtk.vtkTransform()
# tr.Translate(0, 0, -20)
# tp = vtk.vtkTransformFilter()
# tp.SetInputData(msh.dataset)
# tp.SetTransform(tr)
# tp.Update()
# out = tp.GetOutput()
# msh.dataset.DeepCopy(out)
# print(msh.dataset.GetPointData().GetArray("RGBA_TEST"))
# exit()

msh.z(-10)
# msh.pointdata.select("RGBA_TEST") # WORKS HERE
print(msh)

show(pic, msh, axes=8)

print() before msh.z(-10) gives:

vedo.shapes.Grid at (0x56008d6a7c40)
name          : Image
pointdata     : "RGBA" (uint8), dim=4
pointdata *   : "RGBA_TEST" (uint8), dim=3

print() after:

vedo.shapes.Grid at (0x56008d6a7c40)
pointdata     : "RGBA" (uint8), dim=4
pointdata **  : "RGBA_TEST" (float32), dim=3

Single star = active scalars Double star = active vectors.

needs to be looked at, as there no good reason for this to happen. Might be a casting to float32 problem.

@antmatyjajo

[antmatyjajo]

Hi Marco, thanks for looking into it :)

Might be a casting to float32 problem

Seems there is something related to this... Whether it is the casting itself, or VTK's interpretation 🤔 Since I have a few different pointdata arrays in my real data, I thought I'd take a look at what happens to the others and there is indeed some (to me) quite odd behaviour. I adapt your example to show what happens.

Case 1

msh.pointdata["RGBA_TEST"] = arr[:, (0,1,2)].copy()
msh.pointdata["RGBA_TEST2"] = arr[:, (0,1,2)].copy()
msh.pointdata.select("RGBA_TEST2") # select last

Does not work.

Print 1:

vedo.shapes.Grid at (0x1380afff0)                                          
...
pointdata     : "RGBA" (uint8), dim=4
pointdata     : "RGBA_TEST" (uint8), dim=3
pointdata *   : "RGBA_TEST2" (uint8), dim=3

Print 2:

vedo.shapes.Grid at (0x1380afff0)                                          
pointdata **  : "RGBA_TEST2" (float32), dim=3
pointdata     : "TextureCoordinates" (float32), dim=2
pointdata     : "RGBA" (uint8), dim=4
pointdata     : "RGBA_TEST" (uint8), dim=3

N.B., nothing has happened to RGBA_TEST pointdata, it's still 3D uint8.

Case 2

msh.pointdata["RGBA_TEST"] = arr[:, (0,1,2)].copy()
msh.pointdata["RGBA_TEST2"] = arr[:, (0,1,2)].copy()
msh.pointdata.select("RGBA_TEST") # select any that is not last

This does work.

Print 1:

vedo.shapes.Grid at (0x12ee929c0)                                          
...
pointdata     : "RGBA" (uint8), dim=4
pointdata *   : "RGBA_TEST" (uint8), dim=3
pointdata **  : "RGBA_TEST2" (uint8), dim=3

Print 2:

vedo.shapes.Grid at (0x12ee929c0)                                          
...
pointdata **  : "RGBA_TEST2" (float32), dim=3
pointdata     : "TextureCoordinates" (float32), dim=2
pointdata     : "RGBA" (uint8), dim=4
pointdata *   : "RGBA_TEST" (uint8), dim=3

After the transform VTK seems to have

1. taken the last-added pointdata, (only) if it's 3D, and
2. cast that to float and made it an active vector pointdata

So if there's an active 3D (RGB) pointdata, that is not the last-added array in pointdata, everything is fine, but if it's the last one, this bug appears. I think.

[marcomusy]

Thanks! Indeed what happens is that the vtkTransformFilter casts uint8 type into float32 only if a dataset has any "active vector" array present.

I think this has to be an unintended behavior in VTK as it makes very little sense to me.... I could only partially fix this in vedo (just pushed to master).

I guess if the mesh position is set before anything related to data arrays all will be working fine.

[antmatyjajo]

Thanks :)

I guess if the mesh position is set before anything related to data arrays all will be working fine.

That's working great for me.

Indeed what happens is that the vtkTransformFilter casts uint8 type into float32 only if a dataset has any "active vector" array present. I think this has to be an unintended behavior in VTK as it makes very little sense to me....

Understood, and yes, made little sense to me!

But... After trying a bunch of things, maybe it actually does make some sense?

I've been stuck here thinking about a translation because that's what I'm using; indeed the selected array is cast to float, but the values are equal.

But what about rotation and scaling? If an "active vector" that's actually a vector quantity is assigned to a pointcloud and that pointcloud were to be rotated or scaled, well, then the vectors must be rotated or scaled, hence the cast to float.

And, indeed, they are; if we rotate or scale our image mesh, VTK does indeed "rotate"/"scale" our "vector" (RGB) and we end up with funny colours (after ensuring everything is back in 0-255 range).

BTW since Vedo sets an active vector data on assignment into pointdata, whether or not 'rbg' is in the field name, then your fix in master will deal with that case, but what if a user sets their own vector data? That will be unselected on assign, right? Maybe you are already well aware of this, in which case, sorry 😅

[marcomusy]

Yes! Makes sense now! The vectors are cast to float to accommodate for transformations, eg:

from vedo import *

plane = Grid()
plane.pointdata["SomeVector"] = np.zeros([plane.npoints, 3]) + [1,0,0] 

plane.rotate_x(20).rotate_y(20)

n = plane.pointdata["Normals"] * 0.05
v = plane.pointdata["SomeVector"] * 0.05
arrsn = Arrows(plane.vertices, plane.vertices + n, c='red5')
arrsv = Arrows(plane.vertices, plane.vertices + v, c='blue5')
show(plane, arrsv, arrsn, axes=1)