RoboCupAtHome / RuleBook

Rulebook for RoboCup @Home 2024
https://robocupathome.github.io/RuleBook/
Other
149 stars 60 forks source link

[DSPL] NEW Versions of Dell Alienware Laptops #433

Closed max-schoebel closed 5 years ago

max-schoebel commented 6 years ago

So we (B-IT Bots @Home team at HBRS, Sankt Augustin, Germany) are about to buy a Dell Alienware as AddOn to our Toyota HSR and saw that Dell recently released new versions of their Laptops.

The Problem with that is: All Alienware 15 and 17 models with i7 processors are available with a GeForce GTX 1070 only. As soon as you want a GTX 1080 in the laptop, the i9 processor is mandatory in its configuration. However that processor is not allowed according to the rulebook. I don't know if other, more flexible configurations are available in other countries, but at least for Germany the above mentioned "restrictions" hold.

I think the rulebook was written when i9 processors have not been available in the Alienware Laptops yet. I think the rulebook should probably be modified to address these changes. Otherwise no german team, that is going to buy an Alienware Laptop for the next competition is able to use a GTX 1080 in the laptop.

Soo either the GTX 1080s could be prohibited all together (which is of course bad for teams that already bought one of the old Alienware versions) or the new i9 processors could be allowed to make GTX 1080s available for future purchases again. What do you think?

justinhart commented 6 years ago

Intel i7-4790HK. Passmark: 11188 Intel i9-8950HK Passmark: 14548

Nvidia GeForce 1070 Passmark: 11150 Nvidia GeForce 1080 Passmark: 12256

What you're arguing here is that the difference between the 1070 and 1080 is so significant (less than 10% difference) that if we don't allow you to get a CPU that is 30% better than the rest of the teams that this would be unfair, and that the people who already purchased their laptops which comply from the rulebook should be banned from using them in the competition.

Nobody is getting banned from using the previously-approved laptops in competition. If we allow you to purchase an i9 CPU, then we're giving you individually a rather large advantage over the teams who already purchased their laptops.

Similarly, it is not the case that no German team would be able to purchase a compliant laptop. Dell changed their lineup and the 1080/i7 combo is no longer available in the United States either. German teams who already purchased their laptops are not running into this issue. Many teams have already acquired their equipment, and at significant expense, so, what you're actually asking for is an individual advantage for your team.

Why is the i7/1070 solution not acceptable, or simply buying a compliant laptop from an Alienware reseller? I don't know all of the German distributors, but you can get an i7/1080 Alienware laptop identical to the one that my team has purchased from Newegg in the US.

Would you think it fair if you were limited to an i7 CPU while all of the other teams were allowed i9s?

Justin

justinhart commented 6 years ago

I think that there are two policies which would be reasonably fair, in this case

Either for the TC to relax the policy that the laptop be specifically an Alienware laptop, and allow B-IT Bots @ Home to purchase a non-Alienware branded laptop rather than one that exceeds the CPU specifications.

Or for the TC to say that B-IT Bots needs to purchase an Alienware laptop which hits the specification.

If no other TC member comment within 24 hours I will declare an official ruling from one of those two options at that time.

justinhart commented 6 years ago

B-IT Bots, I would like to request that during this 24 hour period you suggest an alternative laptop which meets the specs of i7/1080.

kyordhel commented 6 years ago

I agree with @justinhart in that is not fair that some teams have access to an add-on that has superior performance in a Standard Platform League. On the other hand, I totally understand that nobody wants to buy outdated hardware.

Here I would delegate the decision on the DSPL itself. If all teams with agree in allowing the i9/1080 combo, then you are free to go. Otherwise, I think that hardware can't be used in 2018.

Having a Ferrary doesn't mean you'll push the gas all way down, but you shouldn't be allowed to race against beetles unless all drivers agree (might be a granny competition).

argenos commented 6 years ago

Why is the i7/1070 solution not acceptable, or simply buying a compliant laptop from an Alienware reseller?

@justinhart It is and those are the options we are currently discussing with our purchasing department. However, before making a decision and purchasing anything, it was worth asking what the official policy was with respect to the GTX 1080 to make the investment worthwhile.

I guess the question can be rephrased a bit differently, what is the long(ish)-term policy for the DSPL laptops? Newer teams who join the league might also face these difficulties.

alex-mitrevski commented 6 years ago

In addition to what @max-schoebel and @argenos have said, I would like to add that we will not participate in Montreal, so we are really preparing for next year and the long term; that is the reason why we are bringing this up. We could certainly find a reseller where we could buy the currently approved configuration, but since Dell has already changed their offer, we believe it's worth considering what happens with the laptops in the long run, particularly because, as already mentioned, the equipment is anyway quite expensive.

max-schoebel commented 6 years ago

I am not trying to get an individual advantage for our team here as @justinhart suggests. As this is a rather big investment for us, we just don't want to buy out of date hardware today in order to participate in a competition a year from now. Additionally, this issue will concern anyone who has not yet bought a laptop in a country, where dell does not offer the i7/1080 configuration and not just us. I was merely trying to get the discussion about this issue started.

Furthermore, I think the comparison of @justinhart is a bit flawed: The i9 should be compared against the currently available i7 in Alienware Laptops and not to one that was released four years ago. The i7 8750H gets 12604 on passmark, which renders the i9 around 14.5% better than the current i7.

Additionally, comparing GPUs with that benchmark is probably not adequate in our case, as we are not interested in desktop or gaming performance. I assume the Laptop will be most beneficial and used for vision tasks probably using Deep Learning. In parallel computation on GPUs the number of CUDA cores and speed/size of the GPU memory are the numbers to look for. GTX 1070: 1920 CUDA Cores, 8 Gbps Memory Clock GTX 1080: 2560 CUDA Cores, 10 Gbps Memory Clock Both have 8GB Ram in total.

In my opinion the difference there is indeed significant!

justinhart commented 6 years ago

Is your plan to compete in RoboCup@Home 2018 with your laptop or RoboCup@Home 2019?

The specs for 2019 will be released next year.

The i7/1080 combination was available in Germany. It's no longer available in the US either. Your problem is that you didn't act sooner, not that the league treated Germany inequitably.

justinhart commented 6 years ago

Going through this chain, briefly:

1) The current rulebook in the repo is for 2018, not for 2019. We aren't listing the specs for 2019 in the 2018 rulebook. Yes, the 2019 rulebook will be updated.

2) Dell has the same lineup worldwide. That's why they were chosen. Everyone who got their laptop for 2018 and purchased it already had the i7/1080 combination available to them.

3) I'm not trying to pick a fight, but I am trying to put all of the facts into perspective. The 2018 rulebook applies to the 2018 competition. It is reviewed and updated every year. If you are concerned about your hardware for 2019, then those changes will be in the 2019 rulebook, not the 2018 rulebook. The whole point on capping the hardware is to limit the amount that teams need to spend on this venture, so changing the bar in the 2018 rulebook now would be inappropriate.

4) It was a typo on my behalf to list 4th gen i7 specs. I wanted to make sure that your concern was addressed and had only a few free minutes to do so. Sorry about that.

5) I apologize for jumping the gun on my previous email. I didn't thoroughly read the entire chain and realize that this conversation does not apply to hardware that will compete in Montreal, which is kind of the key factor here. Based on the original chain, I thought I was addressing concerns about 2018 hardware.

6) I sympathize with the need to "future-proof" your laptop. We don't know what the maximum reasonable specs of machines will be next year.

It is obvious that we will need to make some kind of plan to allow for new teams to enter the fold while ensuring that current teams have some reasonable method of assuring that their hardware is up-to-spec, while assuring that veteran teams who have already invested are not just rolled over by new teams. We're not going to be able to thoroughly address that concern with a quick ruling. Just coming up with a reasonable solution for this year took a few weeks. The DSPL should take some time and consider what it means to be within spec.

I would propose that a solution is for the TC to put together a list of machine serial numbers that are certified in-spec. Under THIS solution, the TC could authorize the purchase of hardware that is currently out of spec for the immediately upcoming competition while assuring that future TCs have a method of quickly validating team hardware. This spreadsheet would be viewable by the entire league, so everyone knows what's under the hood of your HSR's backpack. The decision and proposal for hardware would take place on the league list, allowing the other DSPL teams to respond if they think that the hardware offers some sort of unfair advantage.

kyordhel commented 6 years ago

I think this thread is getting a little bit out of context.

This is not about any particular team. The unavailability of the chosen standard laptop affects all current and new teams joining the league. Despite what @max-schoebel said, the TC can't rely on what a team plans to use, but the raw available resources to all (some teams using DL/CUDA might get a big advantage). Hence, this has to be further on discussed within the TC and the current team in the league. Considering the investment several teams made, I find unlikely specs be updated, but let's see.

I think the DSPL needs to run some benchmarking on their own to show TC the performance difference of both HW.

In a quick reply for @max-schoebel, @argenos, and @alex-mitrevski, you will be able to use the i9/GTX1080 Alienware in 2019 for sure, if not as backpack laptop as external computing device.

awesomebytes commented 6 years ago

Other laptops fulfilling the i7+gtx1080 spec:

ASUS GL702VI-MH72 17.3" 60 Hz Intel Core i7-7700HQ (2.80 GHz) NVIDIA GeForce GTX 1080 8 GB VRAM 256 GB SATA SSD 1 TB HDD 16 GB Memory

MSI GT Series GT75 TITAN-058 17.3" 120 Hz Intel Core i7 8th Gen 8750H (2.20 GHz) NVIDIA GeForce GTX 1080 16 GB Memory 256 GB M.2 SATA SSD

Newegg seems to ship globally.

MatthijsBurgh commented 6 years ago

Has there been any conclusion to this issue? Especially the 2019 specs. We are planning to use the HSR in Sydney, but we don't want to wait buying the laptop.

justinhart commented 6 years ago

The conclusion last year was that you needed to get a laptop within the current spec, approved by the TC, if it was not the Alienware laptop. This is to say, you pick a laptop which has a processor no better than an 8th Gen i7 and no better than a GTX 1080. You send a link to the laptop to the TC for approval. If we approve, it's cool. If not, we'll tell you why. We don't plan to increase the specs this year, as not all teams can afford an annual refresh on the laptop.

Justin

MatthijsBurgh commented 6 years ago

@justinhart Thanks. We have the same issue of Dell not providing a intel i7 and GTX 1080 configuration. So we will email the TC for approval.

argenos commented 6 years ago

I just want to confirm, based on last year's discussion, that we are still able to use the new Alienwares as external computing:

you will be able to use the i9/GTX1080 Alienware in 2019 for sure, if not as backpack laptop as external computing device.

justinhart commented 6 years ago

The Alienware laptop in the rulebook is specified for the backpack mount on the HSR for DSPL. There are no plans to disallow this in 2019.

kyordhel commented 6 years ago

@MatthijsBurgh I'd say that this is a League-level decision. If all DSPL participating teams are OK with you using a slightly superior computer, the TC won't oppose (here Justin can say no in advance as UT Austin representative). I guess most teams are not using the backpack to its 100% so a small performance increase wouldn't produce a meaningful difference. Nonetheless, I think it would be fair to cap the hardware of the whole SPL to the same tier. This voting shall be done after the Second CFP confirmation round.

As of now, I would advice to buy the laptop and have it running as external computer. If allowed by the other teams, then you'd be able to mount it, otherwise, you still have wifi.

@argenos Yes, using the i9/GTX1080 as external computer (no backpack) is allowed. You may use even a mainframe as external computer