Grasping

[erolley-parnell]

Grasping

Grasping specific high level tasks/benchmarks

Getting to know my home

The robot must detect new changes in the environment, and update the semantic map of the apartment, within a limited time frame. The task shall be performed by the robot autonomously, though it may include moments of symbiotic interaction with a user in the apartment, e.g., to learn more ab out an object or a location. At the end of the knowledge acquisition phase, the robot must show the understanding of the new environment, namely by handling one of the changed objects.

Possible actions include:

• Closing a door connecting two rooms

• Moving two pieces of furniture from one room to another

• Moving two known objects from one furniture to another

• Placing an unknown object(trash) on the floor

Welcoming visitors

The robot needs to handle a set of known and unknown visitors, who will arrive individually at the home entrance in random sequence. The robot must correctly recognise the known visitors and interact with the unknown visitors to identify them and understand their purpose of visit. The robot must then perform a set of visitor-specific behaviours that could range from manipulating and delivering of objects to guiding and following the visitors.

Possible actions include:

• Opening the door: The front door will be closed. The robot can either use its own manipulators to open the door or request the visitor to open it (e.g. using speech). An extra achievement is assigned for the former case. The below image shows an example of the type of handle that will be mounted on the entrance door.

Catering for Granny Annie's comfort

This task aims at providing general purpose requests of Granny Annie inside the apartment. It focuses on the integration of different robot abilities such as human-robot interaction, navigation and robot-object interaction. After waking up in the morning, Granny Annie calls the attention of the robot by touching a button on a tablet computer. When the robot arrives, she gives a single speech command composing of three actions from the following three categories:

1. Finding an item or a person in the apartment

2. Manipulating and delivering an object

3. Accompanying a person in side or outside the apartment

These three actions are randomly generated into a single command using a command generator in no predefined order. The robot is required to understand the actions and execute them accordingly.

Possible actions include:

• Picking up an object

• Placing the object in a provided location

Visiting my home

This task benchmark focuses on safe navigation in dynamic environments, obstacle avoidance, tracking and recognising a person and human following and guiding navigation modes. In this task, the robot should visit a set of predefined way points while avoiding and/or interacting with different obstacles based on the nature of the obstacle. Furthermore, the robot must interact and follow a previously unknown person outside the arena through a small crowd and then guide that person back to the arena. The following information about the obstacles blocking the path of the robot is provided below:

(Objects will be available to the teams during the setup days)

• Small object: A small Kraft paper shopping bag

• Wheeled object: A rolling chair, cart, wheeled luggage or some other object with wheels.

• Human obstacle: A person who is unknown before hand.

Possible actions include:

• Small object: Remove the small object from the path using the robot's manipulators

• Wheeled object: Gently push away the object to unblock the path using either the manipulators or the robot's body.

• Human obstacle: Kindly ask the human to move away.

For Functional Benchmark

Data Provided by Competition

• 3D models of all objects in digital format.
• Object properties:
  • Single colour
  • On a white background
• Descriptions of all object instances, and the classes they belong to
  • Nature of object communicated through RSBB (Referee, Scoring and Benchmarking Box) (instance and class)
• Reference system for table surface

Sequence of action

• Receive start signal from RSBB
• Return "Prepared" signal
• Receive execution command with target position from RSBB
• Determine the object class
• Object instance within that class
• 2D pose of the object with respect to reference frame on table
• After completion, or error, send end command to RSBB in readiness for next object.

Actions

Summary of Required Actions

• Close/Open doors
  • Input: State instruction, Door state, current pose, goal state for the door, RGB-D pointcloud.
  • Output: Manipulator way points.
• Move furniture
  • Input: State instruction, Furniture instance and class, expected 3D model of object, furniture pose, furniture goal pose, RGB-D pointcloud
  • Output: Manipulator way points
• Move object from one location to another
  • Input: State instruction, object instance and class, expected 3D model of object, object pose, object goal pose, RGB-D pointcloud
  • Output: Manipulator way points
• Place unknown object (trash) on the floor
  • Input: State instruction, RGB-D isolated point cloud of the trash object, world pose of trash object
  • Output: Manipulator way points
• The robot receives the postal mail using its manipulator.
  • Input: State instruction, RGB-D point cloud of package, goal position of package.
  • Output: manipulator way points

Inputs/Outputs

Required Inputs

Robot State instruction

• Indicates to the robot what the goal is, e.g., to reach the starting point for manipulation, to carry out a trajectory, to release an object and return to the home pose.

Door state

• Whether the door is currently opened or closed. Goal state for the door
• What the goal is for the door state (carried with the robot state instruction)

Current pose

• The current joint state of the robot arm, as well as the pose of the robot within the world frame.

RGB-D point cloud

• field of environment around end manipulator for closed loop sensing and obstacle avoidance
  • isolated point cloud of the trash object
  • point cloud of package

Object instance and class

• Of furniture piece (including whether it should be rolled or picked up)
• Of object to be moved from A to B
• The associated expected 3D model of the object.

Object pose

• Estimated orientation and position of object to be moved based on model matching to point cloud.
  • Furniture
  • Object
  • Trash

Object Goal Pose

• The desired end pose of the object to be moved
  • Furniture
  • Object
  • Trash
  • Package

Produced Outputs

Manipulator Way Points

• A series of robot arm states that are fed to the Inverse Kinematic arm calculator.
  • The states consist of angle as well as joint velocities.

Grasp Points

• The pose that the end effector must be at to be able to grasp and pick up the object.

Note: The competition allows re-grasping, if the object is dropped, the robot may make multiple attempts.

[LiamWellacott]

Regarding "Getting to know my home" possible actions. These are not robot actions, this is what might be changed in the room and the robot needs to detect the changes. I think the only manipulation relevant problem in the first task is picking up the changed object.

3D models of all objects in digital format.

Did the rulebook point you to where these can be found? Also are these two sections Data Provided and Sequence of Action missing a heading? it seems like they are the functional benchmark?

[erolley-parnell]

Regarding "Getting to know my home" possible actions. These are not robot actions, this is what might be changed in the room and the robot needs to detect the changes. I think the only manipulation relevant problem in the first task is picking up the changed object.

3D models of all objects in digital format.

Did the rulebook point you to where these can be found? Also are these two sections Data Provided and Sequence of Action missing a heading? it seems like they are the functional benchmark?

This is all I could find in the handbook:

(4.6.2) A set of 3D-printed objects of different shapes will be used for this bench mark. The 3D models of all objects, in digital format, will be available to the teams before the competition allowing them to train their robots in simulation or in real-world by 3D-printing the objects. Furthermore, objects are also provided to the teams during the setup days.

I'll sort the headers