wrong type conversion for struct XrSessionActionSetsAttachInfo

[laurentott]

The OpenXR specs gives the following definition

typedef struct XrSessionActionSetsAttachInfo {
    XrStructureType       type;
    const void*           next;
    uint32_t              countActionSets;
    const XrActionSet*    actionSets;
} XrSessionActionSetsAttachInfo;

but in xr/typedefs.py there is

class SessionActionSetsAttachInfo(Structure):
    def __init__(
        self,
        count_action_sets: int = 0,
        action_sets: POINTER(POINTER(ActionSet_T)) = None,
        next_structure: c_void_p = None,
        structure_type: StructureType = StructureType.SESSION_ACTION_SETS_ATTACH_INFO,
    ) -> None:
        super().__init__(
            count_action_sets=count_action_sets,
            action_sets=action_sets,
            next=next_structure,
            type=structure_type.value,
        )

    def __repr__(self) -> str:
        return f"xr.SessionActionSetsAttachInfo(count_action_sets={repr(self.count_action_sets)}, action_sets={repr(self.action_sets)}, next_structure={repr(self.next)}, structure_type={repr(self.type)})"

    def __str__(self) -> str:
        return f"xr.SessionActionSetsAttachInfo(count_action_sets={self.count_action_sets}, action_sets={self.action_sets}, next_structure={self.next}, structure_type={self.type})"

    @property
    def next_structure(self):
        return self.next

    @property
    def structure_type(self):
        return self.type

    _fields_ = [
        ("type", StructureType.ctype()),
        ("next", c_void_p),
        ("count_action_sets", c_uint32),
        ("action_sets", POINTER(POINTER(ActionSet_T))),
    ]

I had to change both POINTER(POINTER(ActionSet_T)) to POINTER(ActionSet_T) in order to make a successful call to xr.attach_session_action_sets( ... )

[cmbruns]

Thank you @lott999

XrSessionActionSetsAttachInfo.actionSets is a pointer to an XrActionSet. The type XrActionSet is a handle type defined as a pointer to an opaque XrActionSet_T structure. So it's actually correct for action_sets to be annotated as POINTER(POINTER(ActionSet_T)). But it might be clearer if it were written as POINTER(ActionSetHandle)

[laurentott]

Thank you. But then how do I need to fill the XrSessionActionSetsAttachInfo.actionSets field to make xr.attach_session_action_sets( ... ) succeed ? Below I modified the track_hmd.py example to retrieve the controller poses following the openxr_make_actions() and openxr_poll_actions() functions from @maluoi https://github.com/maluoi/OpenXRSamples/blob/master/SingleFileExample/main.cpp The code works if I change POINTER(POINTER(ActionSet_T)) to POINTER(ActionSet_T) in xr/typedefs.py and if I don't I get TypeError: incompatible types, LP_ActionSet_T instance instead of LP_LP_ActionSet_T instance

import time
import xr
import ctypes

# Once XR_KHR_headless extension is ratified and adopted, we
# should be able to avoid the Window and frame stuff here.
with xr.Instance(application_name="track_hmd") as instance:
    with xr.System(instance) as system:
        with xr.GlfwWindow(system) as window:
            with xr.Session(system) as session:

                asci = xr.ActionSetCreateInfo()
                asci.action_set_name = b'gameplay'
                asci.localized_action_set_name = b'Gameplay'
                action_set = xr.create_action_set(instance.handle, asci)
                hand_subaction_path = (xr.Path * 2)(*([xr.Path()] * 2))
                hand_subaction_path[0] = xr.string_to_path(instance.handle, "/user/hand/left")        
                hand_subaction_path[1] = xr.string_to_path(instance.handle, "/user/hand/right")        
                #Create an action to track the position and orientation of the hands! This is
                #the controller location, or the center of the palms for actual hands.
                aci = xr.ActionCreateInfo()
                aci.action_name = b'hand_pose'
                aci.localized_action_name = b'Hand_pose'
                aci.action_type = xr.ActionType.POSE_INPUT.value
                aci.count_subaction_paths = len(hand_subaction_path)
                aci.subaction_paths = hand_subaction_path
                pose_action = xr.create_action(action_set, aci)
                #Create an action for listening to the select action! This is primary trigger
                #on controllers, and an airtap on HoloLens
                aci.action_name = b'select'
                aci.localized_action_name = b'Select'
                aci.action_type = xr.ActionType.BOOLEAN_INPUT.value
                select_action = xr.create_action(action_set, aci)
                #Bind the actions we just created to specific locations on the Khronos simple_controller
                #definition! These are labeled as 'suggested' because they may be overridden by the runtime
                #preferences. For example, if the runtime allows you to remap buttons, or provides input
                #accessibility settings        
                pose_path = (xr.Path * 2)(*([xr.Path()] * 2))
                pose_path[0] = xr.string_to_path(instance.handle, "/user/hand/left/input/grip/pose")
                pose_path[1] = xr.string_to_path(instance.handle, "/user/hand/right/input/grip/pose")
                profile_path = xr.string_to_path(instance.handle, "/interaction_profiles/khr/simple_controller")
                bindings = (xr.ActionSuggestedBinding * 2)(*([xr.ActionSuggestedBinding()] * 2))
                bindings[0].action = pose_action
                bindings[0].binding = pose_path[0]
                bindings[1].action = pose_action
                bindings[1].binding = pose_path[1]
                suggested_binds = xr.InteractionProfileSuggestedBinding()
                suggested_binds.interaction_profile = profile_path
                suggested_binds.count_suggested_bindings = len(bindings)
                suggested_binds.suggested_bindings = bindings
                xr.suggest_interaction_profile_bindings(instance.handle , suggested_binds)
                # Create frames of reference for the pose actions
                hand_space = [None] * 2
                for hand in range(2):
                    asci = xr.ActionSpaceCreateInfo()
                    asci.action = pose_action
                    asci.pose_in_action_space = xr.Posef()
                    asci.subaction_path = hand_subaction_path[hand]
                    hand_space[hand] = xr.create_action_space(session.handle , asci)
                #Attach the action set we just made to the session
                ai = xr.SessionActionSetsAttachInfo()                
                ai.count_action_sets = 1
                p_action_set = ctypes.cast(
                    ctypes.byref(action_set),
                    ctypes.POINTER(xr.ActionSet_T))        
                ai.action_sets = p_action_set        
                xr.attach_session_action_sets(session.handle, ai)

                for _ in range(10):
                    session.poll_xr_events()
                    if session.state in (
                        xr.SessionState.READY,
                        xr.SessionState.SYNCHRONIZED,
                        xr.SessionState.VISIBLE,
                        xr.SessionState.FOCUSED,
                    ):
                        session.wait_frame()
                        session.begin_frame()
                        view_state, views = session.locate_views()
                        print('LEFT eye pose' + str(views[xr.Eye.LEFT.value].pose), flush=True)

                        if session.state == xr.SessionState.FOCUSED:

                            # Update our action set with up-to-date input data!
                            active_action_set = xr.ActiveActionSet()
                            active_action_set.action_set = action_set
                            active_action_set.subaction_path = xr.NULL_PATH

                            sync_info = xr.ActionsSyncInfo()
                            sync_info.count_active_action_sets = 1
                            p_active_action_set = ctypes.cast(
                                ctypes.byref(active_action_set),
                                ctypes.POINTER(xr.ActiveActionSet))
                            sync_info.active_action_sets = p_active_action_set

                            xr.sync_actions(session.handle, sync_info)

                            render_hand = [None] * 2
                            hand_pose = [None] * 2
                            # Now we'll get the current states of our actions, and store them for later use
                            for hand in range(2):
                                get_info = xr.ActionStateGetInfo()
                                get_info.subaction_path = hand_subaction_path[hand]
                                get_info.action = pose_action
                                pose_state = xr.get_action_state_pose(session.handle, get_info)            
                                render_hand[hand] = pose_state.is_active

                                space_location = xr.locate_space(hand_space[hand], session.space.handle, session.frame_state.predicted_display_time)
                                if space_location.location_flags & xr.SPACE_LOCATION_POSITION_VALID_BIT and space_location.location_flags & xr.SPACE_LOCATION_ORIENTATION_VALID_BIT:
                                    hand_pose[hand] = space_location.pose
                                print('HAND ' + str(hand) + ' ' + str(hand_pose[hand]), flush=True )

                        time.sleep(0.5)
                        session.end_frame()

[cmbruns]

What happens if you change your explicit cast like so?

      p_action_set = ctypes.cast(
          ctypes.byref(action_set),
          ctypes.POINTER(xr.ActionSetHandle))

[laurentott]

OK that does the trick, thanks

[cmbruns]

@lott999 I'm glad you got it working, and I'm glad you are working on this.

Your work on this presents an opportunity to improve the pyopenxr bindings.

Anytime an explicit cast like this is required, it's worth asking if the python methods can be improved.

For example, xr.create_action_set() should maybe return an object pre-cast to type ActionSetHandle.

And SessionActionSetsAttachInfo should accept some reasonable values of the action_set attribute without casting or errors.

When the C++ API has two parameters like this for the count and a pointer to an array, a more pythonic interface would be to accept just one parameter taking an object that knows its own length. At the very least a ctypes array constructed like (ActionSetHandle * 1)(*[action_set, ]), but also maybe any valid python sequence of ActionSetHandles, like simply [action_set, ].

[laurentott]

I can avoid the ctypes.{cast,byref,POINTER} using your idea of stacking the action sets in a list, and using that list to get the count.

#Attach the action set we just made to the session
ai = xr.SessionActionSetsAttachInfo()
action_sets = [action_set, ]
ai.action_sets = (ActionSetHandle * len(action_sets))(*action_sets)
ai.count_action_sets = len(action_sets)
xr.attach_session_action_sets(session.handle, ai)

Thanks

[cmbruns]

That's great! That's more semantically reasonable code. I still think it would be useful to push some of that boilerplate down into SessionActionSetsAttachInfo. But also wrap some of this into a higher level ActionSet class, in the same way we are abstracting the other FooHandle classes into higher level classes like Session, Instance, and System.

[cmbruns]

@lott999 I've just finished converting the hello_xr example into python. My working action set code is at https://github.com/cmbruns/pyopenxr_examples/blob/8c7fb4795c49430e80180c2a9872bcb4dd8494b8/xr_examples/hello_xr/openxr_program.py#L335

I'd appreciate any comments or feedback you might have.