QR-Tagged Cubes instead of real Objects

[marcomasa]

One aspect of decreasing difficulty of the tasks (especially for newer teams) will be the introduction of QR-tagged Cubes.

Teams will be able to choose before their run wether they want to do a normal test with the common objects or a simplified version with QR-tagged Cubes.

These could replace the real objects in benchmark scenarios and therefore eliminate the requirement for teams to develop object detection algorithms. As teams still need to develop a robotic system that does navigation, autonomous task handling etc., we think that this could potentially enable some teams to show more performance during a robocup without relying on good vision.

The removal of objects with varying shape and weight simplifies some hardware requirements for the manipulator. We think that the cubes may be 3D-printed (therefore lightweight) and of a fixed size (e.g. 4x4x4 cm). This way teams with cheaper grippers / robotic arms can still handle those objects.

As we still want to encourage teams to develop hardware & vision and do not want to hand out free shortcuts, a penalty factor (e.g. x 0.5) will be applied to all manipulation points gained during the run. The factor is yet to be discussed, as we'd like this to enable teams to score relatively high in the competition (good for sponsors etc.), but also still want the regular competition to be the "usual way of winning".

The TC will discuss the details in the upcoming weeks and publish standardized versions of the cubes and markers that teams can download and use for development.

[AlcatrAAz]

Additional idea: Combine real objects with QR tagged object (instead of replace them all for the run)

This means remove the additional "RockIn obejcts" and replace them with a set of QR objects.

• Every team has still the same run so the performance can be better compared (no need for penalty points)
• Good teams can do both objects to get the maximum points
• Beginner teams can do the QR objects (without advanced vision)
• Every run (BMT, BTT1 ...) contains both set of objects (e.g. 50% real and 50% QR objects)

[leanderb]

I do not really like the idea of combining old objects with QR tagged objects, because it feels like a step back. I really like the idea, that a team can replace all objects with tagged ones, because this will significally reduce development efforts for new teams. The last few years have shown how challenging the start for a new team is - so let's help them a little bit. But the last years also show that object detection can be solved by the teams with amazing results - so there's no need to simplify it as much.

[leanderb]

TC meeting:

• each team can decide whether they use the real objects or april tagged objects before a run
• april tagged objects are a cube with an april tag on top and object type as human readable text on the outsides
• all cubes have the same size of 40mm x 40mm x 40mm point deduction when april tagged objects selected:
• no points for arbitrary surfaces
• 50% deduction for all manipulation tasks (grasping and placing)
• also decoys are QR tagged objects
• PPT: 5x the same special cavity also marked with april tag
• container: april tag on container bottom

[nicoh]

April tagged objects? How does that reflect reality in an industrial settings? IMHO RoboCup@Work is going backwards in development and not forward. When we wrote the first rulebook we included the possibility to make use of markers for navigation. Not a single team used the markers and that is 10 years ago! Nowadays with very performing object detectors (e.g., YOLO etc.) and the possibility to re-train ML-based detectors for @Work objects I wonder whether including tags for objects is really necessary.

[marcomasa]

With increasing difficulty of the vision (arbitrary surfaces, new object set, etc.), we think that it enables teams to participate in the competition for longer. It just takes away some of the workload for especially new teams, as the neural net is "blocking" at least 1 or more students. The rest of the robotic system still has to function, and we could reward teams like that.

I think the ground markers have not been used because there was just no need. Teams were allowed to map the arena, save positions etc. before competitions. If we add a task like exploration, the ground tags would actually be useful, as they would help the robots to identify the tables.

Also, please keep in mind that not every team has experienced members that have been around for the past 10 years. There are teams that "start from scratch" every 3-4 years.

[nicoh]

Indeed. Teams "start from scratch" every 3-4 years and the same holds true for any other RoboCup competition. Yet, other competitions evolve and make progress. Adding markers is not making progress! Every roboticists aware of the state-of-the-art will see this and will wonder why this league is doing this old stuff. If one argues about "entry levels", one should think about tutorials, summer schools, code sharing, etc. Looking at the performance of the last year it seems that most teams are capable to move around (basically to tune and tailor move_base). So, why not making the object perception and manipulation interesting?

[marcomasa]

First of all, this is only optional and will be penalized, so the normal "interesting" manipulation is still the way teams should aim for. But it introduces the students to all problems related to manipulation, except the training of a neural network. You will still need to do the calculations for arm and gripper movements, object management and so on, aswell as reaching the workstation in the first place. So it prepares teams in every other aspect. But most important: A team can still be relatively successful if their team member responsible for vision is not performing well. The way the current scoring system is, a team with bad vision will not be able to score (significant) points, even though the rest of their system works very well.

Making progress, at least for me, can also be taking a step back and reviewing the changes made in the past years. With the discontinuation of the KUKA youbot, teams were facing the challenge of building their own robot aswell. We should be aware of that, as this requires teams to also have good mechanic / mechatronic designers. So our scope went from mostly software to now building a complete robotic system, that then is required to perform tasks that even industrial manufacturers struggle with (atleast partly).

Adding entry levels to the competition to me is NOT about educating the students that participate or handing out short cuts in form of free source code. I want to see solutions from students that they came up with, as it shows actual interest in the topics and really educates them (learning by doing). And while I agree that summer schools would be great thing, I just dont see the people that would host one. But you are welcome to initiate such offers to new teams :)

[nicoh]

RoboCup@Work was never only about just software. It required always the full-stack roboticists otherwise we would have never came up with novel gripper designs and sensing facilities. Even during the time where all teams used the KUKA youBot we saw many different configurations of arm/sensor mountings/placements. The discontiunation of the KUKA youBot is known since many years and since many years teams had a chance to move forward and teams actually did. You are contradicting yourself. You (cf. the TC) say they want to aim for industrial properties (e.g. reliability etc.), well in in industrial setting if the perception is not working, the complete task will not be achieved, why providing these "easy" entry points? Teams will optimize for that, they will investigate how to gain many points even without having a proper perception. I understand your argument, that you do not want to make it too "difficult" for the teams, but there is always the option to introduce difficulty levels through the environment (e.g., doing proper object detection but either on known or unknown surfaces, or for just a specific set of objects, or for specific situations [in containers or outside containers]).

Concerning the summer schools/camps/tutorials, we/I did this several times. See here: https://h-brs.de/de/pressemitteilung/1-robocupwork-camp-der-hochschule-bonn-rhein-sieg and here https://sites.google.com/view/sciroc-camp-gkk-symposium/gkksymposium

[marcomasa]

Mostly != only. Developing a gripper or mounting a camera is not the same as building a complete robot. Especially if teams are on low budget and cannot afford products like UR3 or Evocortex Base.

You are saying that teams moved forward from the Youbot, and that is true for some. I dont know about the state of the Bit-Bots, but other teams with multiple youbots as spare part "graveyard" held on to that system as long as they could. And I do understand that, as porting the code and developing the new system has been a hell of workload for my team (AutonOHM). Our team also had a major setback in 2019 when we used the new robot, going from the best (GO and WC winners) to one of the worst performing teams in just one year. And in that year, the three best performing teams were still using the youbot. Ever since, the teams with actual high scores used a Youbot (except us) (BitBots and RobOTTO 2021, RobOTTO 2022).

If a team decides to optimize for the simple vision for one year, and is even successful with it, I would welcome that tbh. They would not even have to investigate, we (the TC) will post a point table that shows how many points you can get with the penalty. But i think that developing the network in the following year from that (development) point should not be that hard then, but they get the chance to already show performance this year. As I stated multiple times now, this is for beginners that do not want close to zero points at the end of the competition. The points gained with normal manipulation will be sufficiently higher, but are also way harder to achieve. I think from a motivational standpoint, having the easy option helps teams to set foot in our league.

I also dont think i am contradicting myself there. I dont see how the reliability aspect relates to a robot not being able to do something at all. Introducing some simplifications compared to a real world scenario has always been the way to go, no? The rotating table does not resemble the previous conveyor belt, but allowed more teams to perform well, so the TC back then kept it instead of the CB.

We are trying to balance being beginner-friendly and pro-challenging at the same time, which is not an easy task to do. In the end, i want students to have fun and success on every level of experience. And with the 4 year knowledge reset period, we will run out of the experienced teams if we don't do something for the beginners. They will probably lack motivation / funding until they reach the peak.

Regarding your links: Your first link points to an event 11 years ago? Second one is more recent, but I am not sure wether our league knew about it. And even then, the talks are not exactly @work related, right?

Do you want to assist in the process of writing the new rulebook? Maybe you could join a discussion in our Discord.

[AlcatrAAz]

This whole QR tagged object thing is also meant as an entry point for new teams. The reality is that we had 3 teams at the world cup and 4 teams at the german open. The most teams had problems with basic tasks, the winner of the German Open had ca. 600 points... If we would have 15 teams that can all the basics like navigation, detection and manipulation we probably wouldn't do this. With so less active teams you don't have a lot of people that contributing to the league via TC, OC, software (AtWorkCommander..) etc too. And you can't count that these ca. 8 people who are doing the work atm will be here in 5 years.

[nicoh]

@AlcatrAAz I am not questioning the huge contribution by the teams and all the volunteers. I am questioning whether QR-tagged objects is a good idea and I am questioning whether pausing the PTT and RTT is a good idea. I understand that there are "good" reasons concerning team participation specifically for avoiding zero point participation @marcomasa , but that is just one side of the coin. The other side is that by reducing the complexity (QR-tagged objects) or by pausing some benchmarks (e.g. PTT) one can make the league very soon irrelevant. Basically RoboCup@Work goes behind what we initially proposed 12 years ago! And what we demonstrated 2012 in Mexico. As a co-founder of RoboCup@Work who argued and convinced the community together with many others that RoboCup@Work is (a) relevant, (b) ambitious and (c) contributes to advancing the state-of-the-art in robotics its very sad to see this development.

If you believe that is the right path to take, go for it. The current TC/EC is in charge of RoboCup@Work, but IMHO RoboCup@Work will become soon a "student only" competition without impact on robotics research and without impact on robot benchmarking. Please do not get me wrong, I am not saying that there is no value in "student only", but that was never the only goal of RoboCup@Work, we always wanted also to improve the state-of-the-art in robotics research and we always wanted to establish good benchmarking protocols.

[marcomasa]

I think you are focusing too much on the QR part. We have developed new aspects that make other elements of the league more difficult, and still have the option to add challenges for teams. Just because we offer a more easy path in one aspect does not mean we are not ambitious. One could also make the case that object recognition is "solved" on flat surfaces, so why require it in the first place. Other competitions (Audi Autonomous Driving Cup) were implementing similar simplifications (Road Sign Detection replaced by QR Codes) to focus on other areas of research. I think our league also offers other interesting things than object recognition, e.g. task (re-)planning, efficient navigation, world representations etc.

The current plan also foresees PPT integrated into Advanced Transportation Tasks (new version of BTT3), aswell as the Final. In our last meeting we have also decided to keep RTT in a similar fashion. So the tasks are not removed from the league, but done a bit different.

I have not been around before 2016, so I cant really say wether what you were saying about 2012 is true. But I also agree that, to some extent, the level of performance in the league has been higher in the past than it is right now. Even I witnessed "better" robots some years ago than we see now, but some of the features were suited to a rulebook that left room for interpretation and cheap solutions.

I think Covid also hit our league, and we are trying to re-gain team participation numbers. I never said student only. But I think that students who come to robocup, make great memories and maybe get deep into a research topic are the ones that later will work on solving actual state-of-the-art problems in robotics. But in my experience, most students need some time to get to that point, and I am talking 2-3 years here. Not all universities teach engineering focused on robotics, so some teams have to work even more to get there.