scene.Arrow line and head displayed with different order.

[colemanbroad]

Initially arrow lines are drawn underneath the image.

import sys
from vispy import scene
from vispy import app
from vispy.io import load_data_file, read_png

# app.run()
canvas = scene.SceneCanvas(keys='interactive')
img_data = read_png(load_data_file('mona_lisa/mona_lisa_sm.png'))
canvas.size = img_data.shape[1],img_data.shape[0]
canvas.show()
# Set up a viewbox to display the image with interactive pan/zoom
view = canvas.central_widget.add_view()
image = scene.visuals.Image(img_data, interpolation='nearest', parent=view.scene, method='subdivide')

x = np.random.randint(50,650,(200,2))
x = np.concatenate([x,x+np.random.randint(-20,20,(200,2))], 1)

arrows = scene.Arrow(
    pos=x.reshape((400,2)),
    color='white',
    connect='segments',
    arrows=x,
    # method='gl',
    width=3,
    arrow_color='white',
    arrow_size=12,
    arrow_type='curved',
)

vispy.sys_info() gives:

Platform: Darwin-17.7.0-x86_64-i386-64bit
Python:   3.6.4 |Anaconda, Inc.| (default, Jan 16 2018, 12:04:33)  [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]
NumPy:    1.14.2
Backend:  PyQt5
pyqt4:    None
pyqt5:    ('PyQt5', '5.10.1', '5.10.1')
pyside:   None
pyside2:  None
pyglet:   None
glfw:     None
sdl2:     None
wx:       wxPython 4.0.1
egl:      None
osmesa:   None
_test:    None

GL version:  '2.1 ATI-1.68.20'

[djhoese]

I would say that the overall behavior of this and how you've done it is undefined. By that I mean that you have two things to draw on the screen and have them on the same Z level. How this gets drawn is ultimately up to the GPU and while I agree that it is strange that the heads are drawn separately from the line, I don't know if you can consistently guarantee anything different without setting a Z coordinate in the vertices.

What happens if you specify a Z dimension in the arrow coordinates?

[colemanbroad]

Hey thanks for the quick reply. That half works ;) If I set the z coordinate for arrows of -1 (blue) then they are rendered on top of the image. But if I set it at 1 then the line is behind the image while the head is still on top (white)...

[djhoese]

Just as a sanity check, what if you set it to -100/100? Very possible that this is a bug.

[colemanbroad]

Problem persists with -100/100. I have not touched the node.order, so it is set to zero for each. How does order relate to z-value?

[kmuehlbauer]

Plot order was discussed several times, but do not have pointers yet. Will try to dig something up the next day. @campagnola, if you are around, your expertise is needed here, :-)

[djhoese]

Order is the order that the GPU draws thing in the canvas. The Z order specifies where those things are in the canvas. Typically drawing order isn't needed if things are at different Z coordinates/levels.

What version of vispy are you running? If not from github master, maybe try that. It may have some fixes that aren't in the released version of vispy. You could also try the PyQt4 backend as PyQt5 in the current master branch (and released) version of vispy has some bugs.

[djhoese]

Also could you post your new updated code @colemanbroad so I can more easily test some things?

[colemanbroad]

It's vispy 0.6.0.dev0 from github master branch. I don't have PyQt4 installed right now, but will figure that out and give it a try.

[djhoese]

Are your arrows defined properly according to the docstring:

    arrows : array
        A (N, 4) or (N, 6) matrix where each row contains the (x, y) or the
        (x, y, z) coordinate of the first and second vertex of the arrow
        body. Remember that the second vertex is used as center point for
        the arrow head, and the first vertex is only used for determining
        the arrow head orientation.

[colemanbroad]

I think I'm conforming to the arrows requirements. Here is a gist that reproduces it for me... https://gist.github.com/colemanbroad/eea28c9cbd3df3726f31948d08977e8a

[djhoese]

This must be a draw order thing. Try changing the camera to a turn table:

view.camera = scene.TurntableCamera()

Depending on which side of the image you look at, the arrows "below" it still show the arrow heads. But if you look from the side, the arrows are clearly on either side of the image.

[djhoese]

Ok so a couple things I figured out:

1. The arrows probably shouldn't be children of the Image. This forces the drawing order so that it can't be modified. So arr2.parent = view.scene instead. You can then force the order by doing stuff like arr1.order = 1; image.order = 2; arr2.order = 3 which will work for one orientation but not the other, so not great. However...

2. I think the main issue is a conflict between the way the ImageVisual wants the GL state to be set:

        self.set_gl_state('translucent', cull_face=False)

and how the ArrowHead wants it:

        self.set_gl_state(depth_test=False, blend=True,
                          blend_func=('src_alpha', 'one_minus_src_alpha'))

@kmuehlbauer are you familiar with this logic?

Edit: I should have said, commenting out the arrow head set_gl_state makes the arrows show up properly compared to the image, but the arrow heads are "blended" incorrectly with the arrow line (which extends into the head).

[kmuehlbauer]

@djhoese No, not really. If 'set_gl_state' comes into play, it is usually used to define behaviour of that respective visual. AFAIK this might lead to problems using different behaviour in different visuals. This is also something which I do not fully understand. So bear with me. Hope that @rougier can give more insight here?!

[rougier]

Can you try with cull_face=True instead?

[djhoese]

@rougier if I change cull_face=True in the ImageVisual then the image only shows from one Z direction and the arrow heads are still visible.

I looked at the defaults for translucent, the only difference between the two visuals from what I can tell is depth_test. If I force it to True in the arrow head then I get the same output as before (arrow heads mixing with arrow lines, but respecting the image). If I instead force it to False in the ImageVisual, then both arrow heads and lines are shown through the image.

[rougier]

Depth test is definitely necessary for both the image and the arrow. It is used to build the depth buffer and the "order" of drawing things.

For the mixing between line and head, I'm not sure to what you're referring to exactly but this is probably an expected effect depending on the order between head and line.

[djhoese]

Well without changing the gl state, you get a line that leads in to an arrow head and looks like one solid "object". If I change it as mentioned above and the arrow line and head "mix" in a weird way it looks like:

Good example is the arrow head by her chin.

[rougier]

I think this is because the antialising / transparency for the line is applied again the image and not the head. What you see through the edge of the line is Mona Lisa. If you use thicker lines, maybe this would confirm my diagnostic.

[rougier]

You can also try to not write to the depth buffer while drawing the arrows (depth buffer will still be used, but arrows won't write information into it: glDepthMask(false)

[djhoese]

Thanks. I may not have time to try this out today or even this week @colemanbroad, but I'll see what I can do when I do get the time.

[colemanbroad]

No problem. Thanks for your effort so far!

[colemanbroad]

Hi all :) Any updates on this problem?

[djhoese]

No, sorry. I am very swamped at work right now and vispy is not directly part of my job.

[colemanbroad]

No worries, it's not critical. Good luck at work!

[pauljurczak]

Is it intentional that scene.visuals.Text is rendered always on top of the image? Both cases below render on top of the image in my tests:

t = Text("42", pos=[42, 42, 1], font_size=8, color='blue', parent=image)
t = Text("42", pos=[42, 42, -1], font_size=8, color='blue', parent=image)

System info:

Platform: Linux-5.11.0-40-generic-x86_64-with-glibc2.29
Python:   3.8.10 (default, Sep 28 2021, 16:10:42)  [GCC 9.3.0]
NumPy:    1.21.4
Backend:  Pyglet
pyqt4:    None
pyqt5:    None
pyqt6:    None
pyside:   None
pyside2:  None
pyside6:  None
pyglet:   pyglet 1.5.21
glfw:     None
sdl2:     None
wx:       None
egl:      EGL 1.5 Mesa Project: OpenGL OpenGL_ES
osmesa:   None
tkinter:  None
jupyter_rfb: None
_test:    None

GL version:  '4.6 (Compatibility Profile) Mesa 21.0.3'
MAX_TEXTURE_SIZE: 16384

[djhoese]

@pauljurczak I'll be honest I didn't re-read this whole issue and don't remember everything involved. It likely comes down to needing to specify .order = <int> on the Visuals to make sure they get rendered properly. See the related #1165 issue. If you have a specific use case that isn't working please file a new issue.