Cannot convert type to pomdp_py.framework.basics.Action

Hello, we have used these code files for the Multi-Object Search in order to solve a problem for our final project of a course called "cognitive robotics". In our project the problem involves an agent with partial observability (similar to the original files) which is a submarine that needs to find treasure on the bottom of the ocean while avoiding obstacles (at the moment we only did rocks at the bottom of the ocean, we would also like to have fish that swim in straight lines but one step at a time). the submarine will need to move, look, find (so far actions from the original files), pickup the treasure and unload it to a boat that is static at the top of the map (the ocean surface). We have changed all the files to try and represent this problem and we have come to a problem that is not directly in the files we wrote but it seems its inside the pomd-py library files them selves (of course the issue is with the files we wrote but its harder for us to understand what it is and how to fix it). We would love some assistance with this issue, thanks in advance.

We are also including all the files we have changed and used problem.zip

Thanks @Pokemich6 for describing your situation. Looks like an interesting domain for your course project.

Could you be more specific in what issue you believe exists in pomdp-py library files?

we have come to a problem that is not directly in the files we wrote but it seems its inside the pomd-py library files them selves (of course the issue is with the files we wrote but its harder for us to understand what it is and how to fix it). We would love some assistance with this issue, thanks in advance.

Let me try to help you within this thread. That way this benefits others who come across this.

I am not fully sure unfortunately, that is mainly where I think we could use some guidance. Here is the error i get in the terminal when trying to run the problem file:

pygame 2.2.0 (SDL 2.0.22, Python 3.8.8) Hello from the pygame community. https://www.pygame.org/contribute.html <class 'str'> <pomdp_py.algorithms.po_uct.POUCT object at 0x00000226A28F1580> Traceback (most recent call last): File "c:/Users/pokem/miniconda3/envs/cogrob/Lib/site-packages/pomdp_problems/multi_object_search/problem.py", line 322, in unittest() File "c:/Users/pokem/miniconda3/envs/cogrob/Lib/site-packages/pomdp_problems/multi_object_search/problem.py", line 312, in unittest solve(problem, File "c:/Users/pokem/miniconda3/envs/cogrob/Lib/site-packages/pomdp_problems/multi_object_search/problem.py", line 223, in solve real_action = planner.plan(problem.agent) File "pomdp_py\algorithms\po_uct.pyx", line 225, in pomdp_py.algorithms.po_uct.POUCT.plan File "pomdp_py\algorithms\po_uct.pyx", line 233, in pomdp_py.algorithms.po_uct.POUCT.plan File "pomdp_py\algorithms\po_uct.pyx", line 306, in pomdp_py.algorithms.po_uct.POUCT._search File "pomdp_py\algorithms\po_uct.pyx", line 349, in pomdp_py.algorithms.po_uct.POUCT._simulate File "pomdp_py\algorithms\po_uct.pyx", line 269, in pomdp_py.algorithms.po_uct.POUCT._expand_vnode TypeError: Cannot convert type to pomdp_py.framework.basics.Action

can you try and make out what this error means and if it is solvable?

Also I have attached an incorrect problem file earlier, this is the correct one. Thanks

problem.zip

Good, we have a starting point.

When you say

when trying to run the problem file:

what do you mean exactly? Are you running

python -m pomdp_py -r mos

as in the installation docs?

Based on the error message, I suspect that you created a new Action class for your domain, right? Is that inheriting pomdp_py.Action?

I am running the problem file it self (its in the attached zip file), it is very similar to the original file with a few changes so that it takes into consideration the changes I have made to the other files.

Yes, I have added 2 actions to the domain:

import pomdp_py
import math

###### Actions ######
class Action(pomdp_py.Action):
    """Mos action; Simple named action."""
    def __init__(self, name):
        self.name = name
    def __hash__(self):
        return hash(self.name)
    def __eq__(self, other):
        if isinstance(other, Action):
            return self.name == other.name
        elif type(other) == str:
            return self.name == other
    def __str__(self):
        return self.name
    def __repr__(self):
        return "Action(%s)" % self.name

STEP_SIZE=1
class MotionAction(Action):

    SCHEME_XYTH = "xyth"
    EAST = (STEP_SIZE, 0, 0)  # x is horizontal; x+ is right. y is vertical; y+ is down.
    WEST = (-STEP_SIZE, 0, math.pi)
    NORTH = (0, -STEP_SIZE, 3*math.pi/2)
    SOUTH = (0, STEP_SIZE, math.pi/2)

    def __init__(self, motion, distance_cost=1,motion_name=None):
        if motion not in {MotionAction.EAST, MotionAction.WEST,
                              MotionAction.NORTH, MotionAction.SOUTH}:
                raise ValueError("Invalid move motion %s" % str(motion))

        self.motion = motion
        self.distance_cost = distance_cost
        if motion_name is None:
            motion_name = str(motion)
        super().__init__("move-%s" % (motion_name))

# Define some constant actions
MoveEast = MotionAction(MotionAction.EAST, motion_name="East")
MoveWest = MotionAction(MotionAction.WEST, motion_name="West")
MoveNorth = MotionAction(MotionAction.NORTH, motion_name="North")
MoveSouth = MotionAction(MotionAction.SOUTH, motion_name="South")

class LookAction(Action):
    # For simplicity, this LookAction is not parameterized by direction
    def __init__(self):
        super().__init__("look")

class FindAction(Action):
    def __init__(self):
        super().__init__("find")

class PickUp(Action):
    def __init__(self):
        super().__init__("pick")

class Unloud(Action):
    def __init__(self):
        super().__init__("unload")

Look = LookAction()
Find = FindAction()
Pickup = PickUp()
Load = Unloud()

ALL_MOTION_ACTIONS = {MoveEast, MoveWest, MoveNorth, MoveSouth}

I have added the actions PickUp and Unload so that the agent will be able to move the targets to the boat.

Thanks for posting your code. What you did there seems fine.

I looked at the traceback again. The error happens at po_uct.pyx:269. That line numerates actions returned by agent.valid_actions, which calls agent.policy_model.get_all_actions(...) by default.

I am suspecting that the actions returned by your call to agent.policy_model.get_all_actions are not entirely of type pomdp_py.Action.

Could you paste the content of your PolicyModel code especially the get_all_actions function?

Of course, here it is:

import pomdp_py
import random
from pomdp_problems.multi_object_search.domain.action import *

class PolicyModel(pomdp_py.RolloutPolicy):
    """Simple policy model. All actions are possible at any state."""

    def __init__(self, robot_id, grid_map=None):
        """FindAction can only be taken after LookAction"""
        self.robot_id = robot_id
        self._grid_map = grid_map

    def sample(self, state, **kwargs):
        return random.sample(self._get_all_actions(**kwargs), 1)[0]

    def probability(self, action, state, **kwargs):
        raise NotImplementedError

    def argmax(self, state, **kwargs):
        """Returns the most likely action"""
        raise NotImplementedError

    def get_all_actions(self, state=None, history=None):
            """note: find can only happen after look."""
            """note: pick up only the robot is not holding."""
            """note: unload only the robot is holding."""
            can_find = False
            can_pick = False
            can_unload = False
            robot_state = state.robot_state()
            holding = robot_state.holding
            if holding:
                can_unload = True
            if not holding:
                can_pick = True
            if history is not None and len(history) > 1:
                # last action
                last_action = history[-1][0]
                if isinstance(last_action, LookAction):
                    can_find = True
            find_action = set({FindAction}) if can_find else set({})
            pick_action = set({PickUp}) if can_pick else set({})
            unload_action = set({Unloud}) if can_unload else set({})
            if state is None:
                return ALL_MOTION_ACTIONS  | {LookAction} | find_action | pick_action | unload_action
            else:
                if self._grid_map is not None:
                    valid_motions =\
                        self._grid_map.valid_motions(self.robot_id,
                                                    state.pose(self.robot_id),
                                                    ALL_MOTION_ACTIONS)
                    return valid_motions | {LookAction} | find_action | pick_action | unload_action
                else:
                    return ALL_MOTION_ACTIONS | {LookAction} | find_action | pick_action | unload_action

    def rollout(self, state, history=None):
        return random.sample(self.get_all_actions(state=state, history=history), 1)[0]

def adjacent_position(self, state, robot_id):
    robot_pose = state.pose(robot_id)
    rx, ry, rth = robot_pose
    adjacent_positions = [(rx+1, ry),(rx+1, ry+1),(rx, ry+1),(rx-1, ry+1),(rx-1, ry),(rx-1, ry-1),(rx, ry-1),(rx+1, ry-1),(rx, ry)]
    return adjacent_positions

Well, the error is in your code.

Find is an Action object. FindAction is a class.

Notice their difference. Same goes for the other action types.

Ok, thank you I understand. when I try to run it now I'm getting a different message:

pygame 2.2.0 (SDL 2.0.22, Python 3.8.8) Hello from the pygame community. https://www.pygame.org/contribute.html <class 'str'> Traceback (most recent call last): File "c:/Users/pokem/miniconda3/envs/cogrob/Lib/site-packages/pomdp_problems/multi_object_search/problem.py", line 321, in unittest() File "c:/Users/pokem/miniconda3/envs/cogrob/Lib/site-packages/pomdp_problems/multi_object_search/problem.py", line 311, in unittest solve(problem, File "c:/Users/pokem/miniconda3/envs/cogrob/Lib/site-packages/pomdp_problems/multi_object_search/problem.py", line 222, in solve real_action = planner.plan(problem.agent) File "pomdp_py\algorithms\po_uct.pyx", line 225, in pomdp_py.algorithms.po_uct.POUCT.plan File "pomdp_py\algorithms\po_uct.pyx", line 233, in pomdp_py.algorithms.po_uct.POUCT.plan File "pomdp_py\algorithms\po_uct.pyx", line 306, in pomdp_py.algorithms.po_uct.POUCT._search File "pomdp_py\algorithms\po_uct.pyx", line 350, in pomdp_py.algorithms.po_uct.POUCT._simulate File "pomdp_py\algorithms\po_uct.pyx", line 383, in pomdp_py.algorithms.po_uct.POUCT._rollout File "pomdp_py\framework\basics.pyx", line 635, in pomdp_py.framework.basics.sample_generative_model File "pomdp_py\framework\basics.pyx", line 684, in pomdp_py.framework.basics.sample_explict_models File "C:\Users\pokem\miniconda3\envs\cogrob\lib\site-packages\pomdp_problems\multi_object_search\models\transition_model.py", line 25, in sample
for objid in state[1].objects: File "pomdp_py\framework\oopomdp.pyx", line 136, in pomdp_py.framework.oopomdp.OOState.getitem TypeError: raise: exception class must be a subclass of BaseException

This is the transition model:

import pomdp_py
import copy
from pomdp_problems.multi_object_search.domain.state import *
from pomdp_problems.multi_object_search.domain.observation import *
from pomdp_problems.multi_object_search.domain.action import *

class MosTransitionModel(pomdp_py.OOTransitionModel):
    def __init__(self,
                 dim, sensors, object_ids,
                 epsilon=1e-9):
        self._sensors = sensors
        transition_models = {objid: StaticObjectTransitionModel(objid, epsilon=epsilon)
                             for objid in object_ids
                             if objid not in sensors and objid != "boat"}
        for robot_id in sensors:
            transition_models[robot_id] = RobotTransitionModel(sensors[robot_id],
                                                               dim,
                                                               epsilon=epsilon)
        transition_models["boat"] = BoatTransitionModel(dim, epsilon=epsilon)  
        super().__init__(transition_models)

    def sample(self, state, action, **kwargs):
        oostate = pomdp_py.OOTransitionModel.sample(self, state, action)
        state = copy.deepcopy(state)
        for objid in state[1].objects:
            print(objid)
            if objid not in self._sensors and objid != "boat":  # exclude boat object ID
                state.objects[objid].state = oostate[objid]
        state.boat_state = oostate["boat"]  # update boat state
        return state

    def argmax(self, state, action, **kwargs):
        oostate, ooscore = pomdp_py.OOTransitionModel.argmax(self, state, action)
        state = copy.deepcopy(state)
        for objid in state.objects:
            if objid not in self._sensors and objid != "boat":  # exclude boat object ID
                state.objects[objid].state = oostate[objid]
        state.boat_state = oostate["boat"]  # update boat state
        return state, ooscore

class StaticObjectTransitionModel(pomdp_py.TransitionModel):
    """This model assumes the object is static."""
    def __init__(self, objid, epsilon=1e-9):
        self._objid = objid
        self._epsilon = epsilon

    def probability(self, next_state, state, action, **kwargs):
        if not isinstance(state, MosOOState) or not isinstance(next_state, MosOOState):
            return 0.0
        prob = 1.0
        for objid in state.objects:
            if objid not in self._sensors and objid != "boat":  # exclude boat object ID
                prob *= self._transition_models[objid].probability(next_state.objects[objid].state,
                                                                    state.objects[objid].state,
                                                                    action)
        prob *= self._transition_models["boat"].probability(next_state.boat_state,
                                                             state.boat_state,
                                                             action)  # add boat probability
        return prob

    def sample(self, state, action):
        """Returns next_object_state"""
        return self.argmax(state, action)

    def argmax(self, state, action):
        """Returns the most likely next object_state"""
        return copy.deepcopy(state.object_states[self._objid])

class BoatTransitionModel(pomdp_py.TransitionModel):
    """Transition model for the boat object."""
    def __init__(self, dim, epsilon=1e-9):
        self._dim = dim
        self._epsilon = epsilon

    def probability(self, next_boat_state, boat_state, action):
        return 1.0  # deterministic

    def sample(self, boat_state, action):
        return self.argmax(boat_state, action)

    def argmax(self, boat_state, action):
        """Returns the most likely next boat_state"""
        next_boat_state = copy.deepcopy(boat_state)
        if isinstance(action, Unloud):
            next_boat_state['targets_collected'] += 1
        return next_boat_state

class RobotTransitionModel(pomdp_py.TransitionModel):
    """We assume that the robot control is perfect and transitions are deterministic."""
    def __init__(self, sensor, dim, epsilon=1e-9):
        """
        dim (tuple): a tuple (width, length) for the dimension of the world
        """
        # this is used to determine objects found for FindAction
        self._sensor = sensor
        self._robot_id = sensor.robot_id
        self._dim = dim
        self._epsilon = epsilon

    @classmethod
    def if_move_by(cls, robot_id, state, action, dim,
                   check_collision=True):
        if not isinstance(action, MotionAction):
            raise ValueError("Cannot move robot with %s action" % str(type(action)))

        robot_pose = state.pose(robot_id)
        rx, ry, rth = robot_pose
        dx, dy, th = action.motion
        rx += dx
        ry += dy
        rth = th

        if valid_pose((rx, ry, rth),
                      dim[0], dim[1],
                      state=state,
                      check_collision=check_collision,
                      pose_objid=robot_id):
            return (rx, ry, rth)
        else:
            return robot_pose  # no change because change results in invalid pose

    def probability(self, next_robot_state, state, action):
        if next_robot_state != self.argmax(state, action):
            return self._epsilon
        else:
            return 1.0 - self._epsilon

    def argmax(self, state, action):
        """Returns the most likely next robot_state"""
        if isinstance(state, RobotState):
            robot_state = state
        else:
            robot_state = state.object_states[self._robot_id]

        next_robot_state = copy.deepcopy(robot_state)
        # camera direction is only not None when looking        
        next_robot_state['camera_direction'] = None 
        if isinstance(action, MotionAction):
            # motion action
            next_robot_state['pose'] = \
                RobotTransitionModel.if_move_by(self._robot_id,
                                                state, action, self._dim)
        elif isinstance(action, LookAction):
            if hasattr(action, "motion") and action.motion is not None:
                # rotate the robot
                next_robot_state['pose'] = \
                    self._if_move_by(self._robot_id,
                                     state, action, self._dim)
            next_robot_state['camera_direction'] = action.name
        elif isinstance(action, FindAction):
            robot_pose = state.pose(self._robot_id)
            z = self._sensor.observe(robot_pose, state)
            # Update "objects_found" set for target objects
            observed_target_objects = {objid
                                       for objid in z.objposes
                                       if (state.object_states[objid].objclass == "target"\
                                           and z.objposes[objid] != ObjectObservation.NULL)}
            next_robot_state["objects_found"] = tuple(set(next_robot_state['objects_found'])\
                                                      | set(observed_target_objects))
        elif isinstance(action, PickUp):
            robot_pose = state.pose(self._robot_id)
            z = self._sensor.observe(robot_pose, state)
                        # Update "objects_found" set for target objects
            observed_target_objects = {objid
                                       for objid in z.objposes
                                       if (state.object_states[objid].objclass == "target"\
                                           and z.objposes[objid] != ObjectObservation.NULL)}
            next_robot_state["objects_found"] = tuple(set(next_robot_state['objects_found'])\
                                                      | set(observed_target_objects))  
            # Check if any of the observed targets are adjacent to the robot
            adjacent_target_objects = {objid
                                    for objid in observed_target_objects
                                    if (state.object_states[objid].pose is not None\
                                        and math.sqrt((robot_pose[0] - state.pose(objid)[0])**2 + (robot_pose[1] - state.pose(objid)[1])**2) <= math.sqrt(2))}
            # If there is exactly one adjacent target object
            if len(adjacent_target_objects) == 1:
                target_objid = next(iter(adjacent_target_objects))
                del next_robot_state.object_states[target_objid]
                next_robot_state["holding"] = True

        elif isinstance(action, Unloud):
            robot_pose = state.pose(self._robot_id)
            boat_pose = state.pose("boat")
            #if the robot is adjacent to the boat
            if math.sqrt((robot_pose[0] - boat_pose[0])**2 + (robot_pose[1] - boat_pose[1])**2) <= math.sqrt(2):
                next_robot_state["holding"] = False
            return next_robot_state

    def sample(self, state, action):
        """Returns next_robot_state"""
        return self.argmax(state, action)

# Utility functions
def valid_pose(pose, width, length, state=None, check_collision=True, pose_objid=None):
    """
    Returns True if the given `pose` (x,y) is a valid pose;
    If `check_collision` is True, then the pose is only valid
    if it is not overlapping with any object pose in the environment state.
    """
    x, y = pose[:2]

    # Check collision with obstacles
    if check_collision and state is not None:
        object_poses = state.object_poses
        for objid in object_poses:
            if state.object_states[objid].objclass.startswith("obstacle"):
                if objid == pose_objid:
                    continue
                if (x,y) == object_poses[objid]:
                    return False
    return in_boundary(pose, width, length)

def in_boundary(pose, width, length):
    # Check if in boundary
    x,y = pose[:2]
    if x >= 0 and x < width:
        if y >= 0 and y < length:
            if len(pose) == 3:
                th = pose[2]  # radian
                if th < 0 or th > 2*math.pi:
                    return False
            return True
    return False

I think this will be way more complicated to help with but I would much appreciate your help.

It's actually not that complicated. It is a good debugging exercise, actually.

The error

File "pomdp_py\framework\oopomdp.pyx", line 136, in pomdp_py.framework.oopomdp.OOState.getitem TypeError: raise: exception class must be a subclass of BaseException

is due to a Python-specific syntactic bug (that doesn't affect anything of substance). Basically, that line is currently doing raise NotImplemented but it should be either return NotImplemented or raise NotImplementedError. I could fix that later.

The real problem is

File "C:\Users\pokem\miniconda3\envs\cogrob\lib\site-packages\pomdp_problems\multi_object_search\models\transition_model.py", line 25, in sample for objid in state[1].objects:

state[1] triggers a call of __getitem__ which is not implemented. Purhaps you want to do either

state.get_object_state(1)

state.s(1)

instead?

The reason I didn't implement __getitem__ for OOState is because the square bracket may be confused with querying attributes for ObjectState.

Of course, if you really want that square-bracket behavior to get object state, you can inherit OOState and override the __getitem__ function. I find that the .s(objid) call is also simple enough.

I agree that the square brackets can be confusing. The variable "state" right now is this: MosOOState{67: ObjectState(target,(5, 7)), 59: ObjectState(target,(14, 9)), -114: RobotState(robot,(9, 0, 0)|(),False)}

I want to go through the different states in this dictionary and access only the id of the state, in this case it would be first target, then target then robot. I tried to change the code to the options you gave me and now I'm getting this error:

File "C:\Users\pokem\miniconda3\envs\cogrob\lib\site-packages\pomdp_problems\multi_object_search\models\transition_model.py", line 26, in sample
for objid in state.s(1): File "pomdp_py\framework\oopomdp.pyx", line 172, in pomdp_py.framework.oopomdp.OOState.s KeyError: 1

I don't have a lot of experience with dictionaries so I don't know how I can access information and use it to define variables.

thank you for the help.

Hmmm... I thought that 1 is an object id since you had state[1].

Try either 67, 59 or -114 instead of 1 and see what happens (using the function calls I mentioned above).

I don't understand what you are trying to achieve here. Could you give an example?

I don't have a lot of experience with dictionaries so I don't know how I can access information and use it to define variables.

I get this error now (which seems quite similar):

File "C:\Users\pokem\miniconda3\envs\cogrob\lib\site-packages\pomdp_problems\multi_object_search\models\transition_model.py", line 26, in sample
for objid in state.get_object_state(-114): File "pomdp_py\framework\oopomdp.pyx", line 71, in pomdp_py.framework.oopomdp.ObjectState.getitem KeyError: 0

I don't understand what you are trying to achieve here. Could you give an example?

What I am trying to achieve is to define the variable objid to take the id of each of the objects in the state so that I can check if the id is of sensors/boat. the question is how from the library, state which is MosOOState{67: ObjectState(target,(5, 7)), 59: ObjectState(target,(14, 9)), -114: RobotState(robot,(9, 0, 0)|(),False)}, I can go through all the objects and assign the variable to be the object id. For this example the objid will be first target, then target and finally robot

Will close this for now since it's been a while. Feel free to reopen if necessary.

h2r / pomdp-py

Cannot convert type to pomdp_py.framework.basics.Action #26