Closed kyordhel closed 5 years ago
Can we get a list of current DSPL teams so we know whose arm we need to twist to get this handled? A week is a pretty tight timeframe.
I vote to leave it as is. There isn't even current hardware that is better than the spec we made last year. The 2080 isn't available in a laptop form factor yet and we said that 8th gen i7s were good last year. Nothing else was regulated except for the fact that it should be an Alienware, and teams bypassed that. So, any i7 8th gen with a 1080, and we review this again for next year when it's not on some crazy timeline.
I also vote to leave as is as Justin stated where we all agree on the spec of the machine.
Leave it as is is not an option. New teams can't get old hardware. You may, on the other hand, allow them to buy updated HW
There isn't really updated hardware, though. There's no 2080 hardware in the wild yet for laptops. So, if they got a different laptop that matched the spec, it would be fine.
@argenos @YutaroISHIDA @MatthijsBurgh
The rule should be updated because some teams (including us) got an updated hardware (we got a 2018 version with i9 CPU). Personally I think any laptop that fits the bracket should be allowed because:
There are hardly any GPU performance difference between single-GPU gaming laptops. CPU performance may vary a bit but I don't think any team is taxing the CPU anyway. Also the CPU performance is greatly limited by the cooling and battery power.
Dual-GPU laptops are available, but I don't think any team will choose one due to the battery life, weight, price, and limited performance delta over single GPU ones. If someone is still worried about unfair advantage of dual-GPU laptops, we may disallow dual-GPU laptops.
The teams were already allowed to choose less powerful hardware last year (1070 equipped model). I think any hardware should be allowed as long as they don't have unfair advantage over reference ones. Personally we'd like to use lightweight laptop with 1060 GPU instead of 17" Alienware.
Finally, World Robot Summit 2018 allowed for any laptops that fits the bracket.
Just out of question, don't you already have a laptop compliant with the competition as of last year? I think that this move could force most of the league to upgrade. A current-gen i9 is quite a lot better, as is the 2080 GPU, which I just saw that Dell offers. It's still possible to get compliant hardware that keeps everyone in sync and doesn't force upgrades on teams that may not want the additional expense.
Just out of question, don't you already have a laptop compliant with the competition as of last year? I think that this move could force most of the league to upgrade. A current-gen i9 is quite a lot better, as is the 2080 GPU, which I just saw that Dell offers.
We did not use backpack laptop at the last RoboCup, and we purchased an i9 version right after the RoboCup which is technically NOT compliant with the competition as of last year. I remember someone asked whether they can get a new version because the 1080 GPU was only available with i9 CPU option, and everyone said that should be fine in the next RoboCup.
And current-gen i9 is NOT really better than current-gen i7 CPU, at least with Alienware laptop. The laptop is quite terrible in terms of cooling (Actually almost the worst among 17 inchers) and the CPU throttles down a lot after seconds. Apple had the same issue with their i9 macbooks.
Also the new alienware laptops do not have 2080 GPU. They have 2080 Max-Q GPU (thermally limited version, which I guess is not more powerful than 1080) in much thinner and lighter chassis (which will make cooling EVEN WORSE compared to 2018 models)
So in short, I don't think any team can have unfair advantage however hard they may try, unless they use huge dual-GPU laptop with custom battery pack (Which won't fit in the bracket anyway)
This was the ruling regarding the i9.
"In a quick reply for @max-schoebel, @argenos, and @alex-mitrevski, you will be able to use the i9/GTX1080 Alienware in 2019 for sure, if not as backpack laptop as external computing device."
No commitment was made that i9 Alienware laptops would be allowed to be used as the backpack laptop. No team was ever authorized for that.
====
Here's the benchmark between the i9 9900K and the i7 8700K: https://www.cpubenchmark.net/compare/Intel-Core-i9-9900K-vs-Intel-Core-i7-8700K/3334vs3098
The i9 is 26% better. I'm sure that some company makes a laptop with an adequate heatsink/fan combination to cool this.
====
There is a laptop 2080 that is not the Max-Q 2080, regardless of what is installed in the Alienware.
The 2080 laptop version gets a 3DMark of 9219, on the first benchmark I found https://www.notebookcheck.net/NVIDIA-GeForce-RTX-2080-Laptop-Graphics-Card.384931.0.html
The 1080 laptop version gets a 3DMark of 6547 on the same test https://www.notebookcheck.net/NVIDIA-GeForce-RTX-2080-Laptop-Graphics-Card.384931.0.html
Even if they scored similarly on 3D graphics, the 2080 is on the Turing architecture, so this is a reasonably large shift, and Turing GPUs process neural networks more efficiently.
====
The point of standardizing the laptop was to keep the platform "standard" since DSPL is a "standard platform league". Even if you can't get compliant hardware from Alienware, you can certainly get it from elsewhere.
The current ruling from the TC (which I did not make, I was not able to make yesterday's TC meeting), was that if the entire DSPL does not agree on this, no laptops will be allowed in the 2019 competition. While I am sympathetic to your position (I was the one who originally proposed the laptop backpack), I am very concerned that the league will fail to come to consensus on this, and that the result will be that the laptops are prohibited in 2019, or possibly permanently.
One or two generation's worth of hardware differences is marginal; can the league identify any discontinuities in what's possible given low-XX% improvements in GPU or CPU? Why do we think they exist? I think any difference will be drowned out by the noise of the competition process.
This is a shit-sandwich though. Allowing any laptop is not sustainable for more than this year, and already undermines the league. Forcing upgrades is a nice annual $3K slap in the face, and is more churn than is really required because long term hardware availability can't be guaranteed. Hardware regression will damage the league too.
I don't have a stake with a team, but I do hope someone takes a stand against the backpack. It was a poor bandaid to begin with and should not have been adopted. It's a burden to the league, and hobbling along isn't an effective way to force a better solution.
I don't want to be a troll, and I'm not in the DSPL league. But in SSPL we deal with a platform that I believe is way more underperforming (Atom 1.6Ghz 4core, 4GB RAM plus other annoyances like no root access and an esoteric OS), and teams still compete and do some stuff. Deep learning stuff runs on the external computation device. It's standard.
You can use anything you want as an external competing device tho. (With the WiFi implications that that brings ofc).
Not in the DSPL/SSPL League.
'external device' is even more stupid than some backpack -> you can have a server farm somewhere. In any 'standard' League the Device(s) should be defined and attached to the robot. I agree that any additional device is a burden to the League.
Is the hardware to weak for your solutions? -> compete in open league
The bad bandaid version should have been something that is available for more than 1 year. Business Line Laptops that you can still Buy Refurbished, e.g. Dell Precision M4800 we have in our lab are >5 Years old and you can still get them of Amazon.
edit: SSPL -> All
Relatively off topic: External devices are for all leagues, not for just SSPL.
Please keep a respectful, friendly and nice environment. Remember that some of the most remarkable human discoveries were made by mistake, because we screwed up big time. I'm looking at you, beer.
The main idea for bringing both SPL was to promote code and solution sharing. Having similar hardware will help you to test other team's approaches. You're not rivals, but allies in domestic robotics research
Please remove tailing content when replying from an email client. @awesomebytes @justinhart I'm looking at you.
@Home wise, standard means that teams A and B can swap hardware and there will be no noticeable difference in performance for either of them (maybe TC should implement that for Sydney). This has two advantages:
We feel a bit disappointed that we were limited when buying our laptop. But the opinion of Tech United Eindhoven: We think that we should keep the backpack. Using the robot without a backpack is putting too much pressure on the Wi-Fi, as computing everything on the robot is not possible in our opinion.
As the specs of the laptop is not that limiting at the moment, Tech United thinks that all laptops that fit in the bracket should be allowed.
On the topic of the Wi-Fi extender, it should be allowed without any limits of specs. Because no limits are put on the specs of the Wi-Fi cards of the laptops.
This is maybe destandardizing the league a little bit, but the best way to keep everybody happy.
If you think that you are losing because you don't have the most powerful laptop, you don't understand robotics IMO.
No commitment was made that i9 Alienware laptops would be allowed to be used as the backpack laptop. No team was ever authorized for that.
We are very willing to follow the consensus on this issue. we can either get another laptop or try swapping our current laptop with i7 version. But personally I think the requirement doesn't make too much sense as the @Home league does allow for arbitrary external computing device, or even connecting to remote server farm.
The i9 is 26% better. I'm sure that some company makes a laptop with an adequate heatsink/fan combination to cool this.
Some company does make gaming laptop that can handle mobile i9 with 100% turbo speed (Gigabyte for example). But dell is not one of them for now. What I am saying is that their 2018 and 2019 alienwares are not practically better than the official laptop based on my personal testing (2017 version of alienware 15/17) and allowing them won't give any team unfair advantage or disadvantage.
The point of standardizing the laptop was to keep the platform "standard" since DSPL is a "standard platform league"
As I have said, we already allow for non-standard external computing devices. Why we should standardize the backpack laptop, while allowing for the non-standard external computing devices?
I don't have a stake with a team, but I do hope someone takes a stand against the backpack
I strongly disagree. Without the backpack, all teams need to rely on the unreliable wireless connection - which will make the wifi situation even worse. We had a huge wireless issue last year even with backpack laptops. Imagine what will happen if all teams dump packets through wifi.
Also one more remark - participating at the robocup is never cheap. Registration, travel, robot shipment etc quickly adds up. Spending $3K every other year (or less) is not that much burden IMO.
If the laptop is to remain, then as some of the others have brought up, there is a need for a longer-term policy on the allowed hardware (does this belong in a separate issue? or is this what we are trying to agree on within this week?).
Just like @SJ-YI, we currently have the i9/1080 combo, but we wouldn't want to impose an upgrade on other teams. So for this year, and because we have to reach a consensus in one week, we (b-it-bots) are on board with the following options:
P.S. @kyordhel could you number the viable options? we could then refer to them by number
[...] there is a need for a longer-term policy on the allowed hardware (does this belong in a separate issue? or is this what we are trying to agree on within this week?).
Let's not get off-topic. If the whole DSPL league agrees on a hardware upgrade, we immediately proceed to define the specs. Otherwise, I think it would be better to hold the decision till august-november.
Whatever the outcome is, the TC does NOT want to get involved, even less impose its will. It is up to the league to organize itself and come out with a solution.
A new laptop every year isn’t a financial problem, but it is a benchmarking problem. Having (essentially) a new standard platform every year will make it difficult to assess the league’s progress, especially because it’s possible that a new year’s laptop is worse than the previous in some crippling way.
The vendor needs to provide a loose roadmap so that the TC and the community have a proper chance to plan league development. What happens when HSR2 comes out in a year and has a 1070ti baked in? Does the league drop the backpack, or is it in a difficult position again because some laptops have better GPUs and we don’t want the hardware to regress? What if the GPU becomes backpack only, and the league has to solve standardization long term? Different decisions this year will be more or less compromising in a future that we don't know about.
Rejecting the laptop is a bitter pill, but with the information available, it is the only choice that shakes the league out of a compromising position.
A new laptop every year isn’t a financial problem, but it is a benchmarking problem.
Bear in mind that @Home league already allows for nonstandard external computing devices. For now no team seems to use a crazy powerful workstation yet (i.e. one with 4 tesla GPUs) but nothing prevents teams from using one. That means the GPU performance can differ more than 10X between teams.
Rejecting the laptop is a bitter pill, but with the information available, it is the only choice that shakes the league out of a compromising position.
If we follow that logic, we should ban a) all external computing devices and b) all cloud resources as well because they are not identical. That will be the end of the league, as no teams will be able to do anything interesting and anything relevant anymore.
Also, we cannot make everything absolutely fair anyway. Some team may have a dozen PhDs working on the project for full time. Some team may have just a handful of undergraduate students always busy with other stuff. Do you think it is fair match as long as both teams use identical laptops and robots? Personally I don't think so, and I don't care. RoboCup has always been a code-sharing and solution-sharing event rather than strict competition.
I really don't want to intervene this often.
@Home wise, standard means that teams A and B can swap hardware and there will be no noticeable difference in performance for either of them (maybe TC should implement that for Sydney). This has two advantages:
I have a couple of thoughts on the long-term agenda stuff, but I'll keep my comments limited.
@Nick Walker I get where you're going with this, but I don't think that TRI is likely to release those internal documents. The initial laptop backpack didn't come so much from a proposition on my part to put on a backpack, but a request for an update cycle on the HSR hardware. So, I was asking for a refresh of the internal components across the entire platform. I inquired about the update cycle at that juncture, so my initial ask was really similar to what you're requesting.
The real person to talk to on this would be @Allison Henry, since she is on the TC and works at TRI. I agree that a roadmap would be way way better. It just seems like the sort of thing that TRI is unlikely to provide us with.
@justinhart thanks for tagging me here.
First, I should disambiguate Toyota Research Institute from Toyota Motor Corporation, who makes the HSR's. TRI supports TMC's robot, the HSR. We are working on our own robots and have a generally different design cycle than TMC. TRI is about to get our first next gen HSR's. I don't have one yet at my lab.
I do not know what plans there might be to upgrade the hardware of the existing fleet of the current generation of HSR.
The decision about the laptop was based on lots of teams complaining about the native hardware of the HSR.
Justin, were you involved in the selection? Do you know how it came to pass?
I will ask the other folks from TRI who might be able to get to the bottom of this, some of whom came to us through TMC and designed the HSR.
I just want to see the DSPL teams empowered to perform at the top of their game and to have whatever tools necessary to do so. I sincerely hope you come to an agreement that allows for external compute.
The initial selection of the laptop was based on 3 competing thoughts.
1) Provide a maximum spec. All teams meet the spec. 2) Provide exact hardware with worldwide distribution. 3) Provide no spec.
3) was considered off the table because it broke the notion of a standard platform. It also was axed because teams couldn't agree on the maximum spec. There were teams saying that if they could afford SLI on Xeon machines, they shouldn't be limited, and other teams saying that they shouldn't have to pay an additional tax annually to participate in RoboCup@Home.
1) actually ended up being just frustrating because of argument over the spec. I suggested 1070s on i7s 1 generation behind in order to shave off the extra cost of getting top-of-the-line gear.
2) ended up happening because after argument over the spec the TC just said, "Just pick 1 laptop and everyone gets it."
I'll note that what actually happened was 1). Alienware is a premium brand and schools found laptops with identical hardware for like $1000 cheaper based on cheaper regional brands.
I am told that TMC provides the ~laptops~ backpacks upon request to teams with HSR. Additionally, most teams are leasing the robot for free. I think it is misrepresentative to describe TMC as the vendor when they are giving universities robots, not selling them, unless the universities wish to publish their own research without stipulation based on the HSR, in which case they may choose to pay for their lease.
In summary, ask TMC if you want a ~laptop~ backpack and don't have one. To the best of my understanding, everyone should be able to get the standard laptop without problems. I suspect some people just haven't asked.
Edit: I am not unclear on whether my colleague meant the laptop or just the backpack to hold the laptop and am requesting clarification.
Do you know the laptop specs and can you tell us who to email?
Hello,
following the discussion, as a standard league, at the moment we go with:
But with the conditions as above:
1) Provide a maximum spec*. All teams meet the spec.
So teams will update their hardware as necessary (if they need it), but within a limit to consider standardisation.
Regards, Luis er@sers Team. PUMAS Team
If toyota can provide standard laptops to teams in need, case closed.
But again, can anyone answer my question: why we need to standardize backpack laptop for DSPL alone, while @Home league allows for non-standard external computing device? Price and performance difference between desktops can be vastly larger than laptops. but we do allow them!
Allowing for any laptop has following advantages:
- If your equipment gets broken, another team can help you by lending you theirs
- You are 100% sure you have the resources to use a software package from any other team
And such scenario is far from realistic because:
It's a bandwidth and reliability thing. I have yet to attend a RoboCup where the wifi works properly. That's why the whole "setup your own wifi router" thing is so contentious.
I didn't really tune in for that debate, but, I'm kind of coming around on that. Why did people object to everyone setting up their own WAP?
Bear in mind that @Home league already allows for nonstandard external computing devices. For now no team seems to use a crazy powerful workstation yet (i.e. one with 4 tesla GPUs) but nothing prevents teams from using one. That means the GPU performance can differ more than 10X between teams.
What's preventing them is that the utility of 1 Tesla is very close to the utility of 0 Teslas (because the network won't work anyway). If this were not the case, there'd be just as much kvetching.
In practice, I still think the utility of a laptop with a 1080 is close enough to that of a 2080 that situation is the same; no one really has a basis to complain because it won't impact task performance meaningfully. I am much more concerned about the league being in a weird spot eventually when, say, the 2180 is out and it is a big deal. This is likely to happen. I don't think the utility of external compute is going to go up any time soon.
Opinion of Hibikino is nearly the same the same as Tech United.
We think that we should keep the backpack. Using the robot without a backpack is putting too much pressure on the Wi-Fi, as computing everything on the robot is not possible in our opinion.
@justinhart
I have a couple of thoughts on the long-term agenda stuff [...]
Could you please outline them and send it to us?
@SJ-YI
But again, can anyone answer my question: why we need to standardize backpack laptop for DSPL alone, while @Home league allows for non-standard external computing device?
Historically, WiFi is unreliable. From the very beginning of RoboCup@Home external computing has been allowed, yet barely used due to connectivity problems. You can guarantee 99.9% availability of the resources you have on the other side of an Ethernet wire, but not through wireless connectivity. It is not only my experience. You can ask league's founder Tijn van der Zant, as well as the most experienced members such as Luca Iocchi, Sven Wachsmuth, Komei Sugiura, and Jesus Savage (to name some) and they will tell you about the how every year at least one team got screwed during the open challenge when trying to use external devices. This situation wouldn't change till 2017 and thanks to the outstanding work of Komei who set a new high-quality standard for the league. Yet, it is far from perfect.
In addition, we have observed over the years that external devices offer little advantage to and only a slight increase of performance, while embedded hardware can be a decisive factor to gran a team's victory. Our referees have seen countless time a robot succeed 10 seconds after the timeout, hence not scoring. For a 5 minutes test, these 10 represent only 3.3% of the time and often are wasted in object recognition and planning manipulation. In these circumstances a performance increase of 5% would have caused a robot to score. If you know the basics of control, you'll understand that a eye-hand closed control loop can't reliably be performed over a wireless network without having an adequate model of it.
Furthermore, you're living in a country which has one of the most reliable internet connections in the world according to Akamai, Statista and Speedtest. This reality is not the same worldwide. Teams from countries out of the Top10 will refrain from depending on these devices. On top of that you need to consider that the RoboCup Federation gives priority to rescue and simulation leagues for the use of network since they depend on it to operate. This means that you won't have in the WorldCup the same conditions you have in the lab. The TC is quite serious about it, while also taking into account reliability problems due to channel pollution and low latency.
Concluding, experience dictates that there is a huge difference between external and embedded computing devices, or as @justinhart said: "It's a bandwidth and reliability thing".
Finally, the idea of this issue is to reach consensus on which strategy to take to help all teams in the league, not convince the TC of banning the backpack. If your team managed to solve the wireless connectivity problem of external devices, share the knowledge. Remember our second rule: fair play.
Historically, WiFi is unreliable. From the very beginning of RoboCup@Home external computing has been allowed, yet barely used due to connectivity problems.
I have attended RoboCup since 2010 (albeit in other leagues) and perfectly know how bad the wireless can be at the venue. (I remember seeing ping time of several seconds) In addition, our team was the only DSPL team that did not use a backpack laptop and relied upon external computing device last year.
In addition, we have observed over the years that external devices offer little advantage to and only a slight increase of performance, while embedded hardware can be a decisive factor to gran a team's victory.
I can argue that using newer generation of laptop will not be a decisive factor either. Let's assume that 2080 GPU is 50% more powerful than 1080 GPU. If the object recognition step runs at 10Hz with 1080 GPU, it will run at 15Hz with 2080 GPU - or the team with 2080 GPU will have 0.03 second of advantage over teams with 1080 laptop.
Furthermore, you're living in a country which has one of the most reliable internet connections in the world according to Akamai, [Statista]
I am not saying that the external computing device is good enough to replace the backpack laptop (I'm saying the opposite in fact). What I am saying is that strict standardization of backpack laptop is not logical as it contradicts the current rule allowing arbitrary external computing device.
Finally, the idea of this issue is to reach consensus on which strategy to take to help all teams in the league, not convince the TC of banning the backpack.
I totally agree, and am willing to participate.
[...] perfectly know how bad the wireless can be at the venue.
You asked for an answer, I gave you one from the TC, take it or leave it.
[...] our team was the only DSPL team that did not use a backpack laptop [...]
Backpack laptop use was compulsory to keep balanced the mechanical performance of the HSR in entire league. Thanks for letting us know. We will be more strict in the future.
What I am saying is that strict standardization of backpack laptop is not logical as it contradicts the current rule allowing arbitrary external computing device.
The Executive Committee will discuss the ban of external computing on DSPL in order to foster standardization upon request of at least one of the participants.
Hello,
Given that the rulebook stablishes "3.6. External computing. Robots are allowed to use some form of external computing, for example in the form of so-called 'Cloud services' and/or 'Internet API's' etc.", the back pc seems irrelevant and therefore any should be allowed.
However, after a discussion, and with the aim of boosting research, we would like to suggest the ban of any local-network-based external computing and limit the wifi to the built-in both in the robot and laptop (as in any real-world case scenario).
I think many teams (including us) work on the basis that the hardware is limited to the robot itself and the back pc. If a given solution requires ultra hi-speed connectivity and external computing (or cloud computing), maybe it is not aimed to autonomous mobile robotics yet, and alternatives should be developed. Although, like any other year, those high computing solutions can be shown in the open/final demonstrations.
As always, we are open to discussion (up to February 6th, hehe).
Regards, Luis
I feel like teams are just proposing whatever will help their team the most.
Therefore, I would like to propose that all external compute devices must be authorized by the UT Austin Department of Computer Science and require a University of Texas EID to be logged into. Also, only UT Austin Villa @ Home is allowed to have an external compute device.
Ok, this discussion seems to not really go anywhere. The TC requests PRs in regard to this topic. Further discussions can be had about the specific proposals. The TC will vote on all proposals made as PR in the next meeting.
So lets summarize:
Do you think that slightly newer hardware is making you win/loose the competition? --> Vote for laptop with strict spec regulation. Do you think that specs of the laptop are not that important, but your approach and research will be stuff determine you will win or not. --> Vote for any laptop Do you think your HSR and the best HSR you could think of at the moment, could operate without extra computing power --> vote against any laptop
Do you think that a possible slightly better Wi-Fi connection is making you win/loose the competition? --> Vote against extra Wi-Fi receiver. Do you think that Wi-Fi is not that important or that a possible slightly better Wi-Fi is not making any difference in the competion --> Vote in favor of extra Wi-Fi receiver.
I'm quite surprised on the DSPL community response, although I can't say is pleasant.
FYI, I want to share a thought we got from the Trustees:
Soccer SPL has been working for years with the same hardware. Teams cooperate and and the league improves every year without relying on WiFi. Why can't @Home do the same? Is the most expensive league, the most demanding, and ranking bottom in teamwork, code sharing, and paper publishing.
Can it please be clarified what option 3 and 4 are in the poll as the description of (new) and (any) do not match the description provided in the text above the voting table
The poll is closed now and its outcome irrelevant. TC just learned that DSPL is unwilling to cooperate as a whole and consensus is not possible since each team only pursues self interest, leaving aside collaboration and community building.
If you have a possible solution, please send a PR so the TC can evaluate it.
Aren't we supposed to reach a unanimous decision by Feb 6th? I thought we would discussing over this topic over a week and then make everyone on the same page. And why should presenting each one's opinions be blamed as selfish act?
Soccer SPL has been working for years with the same hardware.
That is actually not true from my experience. Nao robot went through a number of versions and not all the teams could get the latest version of Nao right after the release.
@johaq @balkce
One-liner
DSPL teams have 1 week to reach consensus or Backpack use won't be allowed in Sydney.
Long version
Context
This year we had two requests from new members of the league:
The main purpose of a Standard Platform is to discard hardware variables to make the competition as fair as possible. This would normally mean that no additional hardware can be tethered to the robot. Nonetheless, the Technical Committee made an exception past year, on the condition that the technical specifications for the backpack laptop remain valid for at least 3 years.
Due to manufacturer decisions, this wasn't possible, placing new competitors in a potential situation of advantage or disadvantage. In the meeting of Monday, 28th of January, 2019, The TC has decided to allow the Domestic Standard Platform League to come up with an unanimous decision.
Options
Viable options are as follows:
Wireless receiver
¹ The receiver authorized by the TC would be a USB-powered one with technology 802.11ac / AC1900 or better. ² A custom USB powered mico-switch would be allowed to connect the receiver and the laptop to the robot.
Mechanics and deadline
Consensus must be reached by an unanimous decision of all DSPL teams (attending to Sydney or not), who must place their final vote in this issue no later than Wednesday 6 of February, 2019.
Current TC decision is to rollback to the robot's factory hardware only (same as with all SPL in RoboCup). It will take only one abstention or vote against to validate this decision.
Poll
Only DSPL team leaders are allowed to vote (one per team). None DSPL teams voting will be disqualified. If numbers don't match, all additional hardware will be banned.
Vote here
Edit
Bot wasn't working. Moved to doodle