TC Meeting Minutes 29.08.19

[nickswalker]

Task Cut decisions

• Stage 1 Housekeeper
  • Groceries
    • Consensus is to keep
  • Some in favor adding mandatory doors
  • Clean up vs take out the garbage
    • Garbage was very audience friendly, but a little easy
    • Clean up had some participation, and aligns with Tidy task in Toyota World Robot Summit event
  • Straw poll goes for clean up 6-3
    • We can consider adding an element of garbage to the task
• Stage 1 Party host
  • Except for stand out teams, interactions haven’t gone as expected
  • Carry my luggage kept by consensus
  • Strawpoll goes for Receptionist over Hand Me That (brought down from stage 2) 5-4
• Stage 2 House Keeper
  • Clean the table kept by consensus
    • Some good manipulation from teams already
      • Is it solved?
      • Handling cutlery, detergent pod is still hard (though DSPL is doing well)
      • But teams are largely just dropping things into the rack
    • Hard for SSPL
      • Open for proposals for alternatives targeting this league
      • Harder version of clean up could be introduced as alternative?
    • Opportunity for sponsorship in the future
    • Could be made harder
      • Transparent objects (standardized
      • Other objects on table
• Stage 2 Party Host
  • Proposal: Play Chess (PR open)
  • People like this task
  • But we can’t add totally new for this year
  • Hand Me That
  • Consensus choice from prior discussion
  • Strawpoll goes in favor against Play Chess 5-2
  • Too easy as is
    • Could be made more open

      Roadmaps

• Purpose: long term planning
  • Keep tasks within reach for teams
  • Help us avoid things that are too easy
  • Help us keep hard research out of Stage 1
  • A rule of thumb: A master’s student should be able to get a thesis out of addressing open challenges in a Stage 2 task
• Should balance what people want with what researchers see coming
• Plan forward
  • Add onto Johannes’ and Luis’ documents
  • Integrate references
  • Johannes will email documents to TC, EC

    Object Set standardization

• YCB
  • Positive experience at Japan Open where only one set was available
  • Teams were able to train largely before hand
  • Data available for free to teams
  • Amherst was having trouble completing payment from Japan
  • They claim it will be resolved by September
  • Agreement that we’ll adopt YCB
    • Nick will make pull request
  • Ikea dishes and cutlery
  • No existing high quality data, so looking for contributions here

Task time limits

• Some tasks take more time than others to get more than 0 points
• Will be too hard to run competition if we allow different timings
  • Scheduling
  • Scoring gets harder with time penalties
• Don’t want to reward partial completion and thereby incentivize teams to partially complete task just to get time bonus
• Refbox will communicate time so teams can better handle
• Less of an issue if we design tasks to fit
  • Rule of thumb: design task for human to take 1 minute, expect robots to take 5x as long

How to handle HRI

• Robots are saying things and expecting referee to do things without clear instructions
  • Stepping through scripted dialogue with no contingencies for things going at different pace or going wrong
• Is this a problem with task, or with how we are reffing?
• Resolved: We’ll put together short guidelines for how refs should interact with robots so we can have a minimum standard for what to expect from the robot
  • Aimed to be on Wiki to avoid rulebook bloat

[nickswalker]

As always, edits and additions from those in attendance (I think all of TC + Mauricio and Alex), and comments from anyone, are welcome