search

odota / core

Open source Dota 2 data platform
https://www.opendota.com
MIT License
1.51k stars 303 forks source link

Desktop client application #137

Closed howardchung closed 9 years ago

howardchung commented 9 years ago

Build a desktop client (Java?) that enables users to parse their own replays, and maybe uploads the results to us

DarkMio commented 9 years ago

If it is going to be a full Java Applet (with JavaFX possibly?), I am happy to help.

Personal parsing with uploading to the database (security, validation?) would offload a lot of traffic.

albertcui commented 9 years ago

Yeah that would be awesome. None of us actually have any experience writing desktop clients.

howardchung commented 9 years ago

Nick says he might want to do it over the summer. Dunno what timeframe we're looking at for this.

Nick's plan was not to include upload, but just have the client parse replays and generate HTML pages for personal use.

If we upload, there's really no good way to authenticate since we can't put a shared secret in a distributed desktop app.

I had a cool idea to make it peer-to-peer so that consensus on a given replay can be achieved, but that's probably way more complicated than it needs to be and won't have a big enough userbase to work.

DarkMio commented 9 years ago

There are different topics to discuss:

1.) Desktop Client with HTML Elements: Deploying an app with the standard-set of libraries is easy as cake. With enough planning, this should be portable to Android aswell! As you said, parsing matches and HTMLing its Data could be done over staged systems:

  1. Parsing
  2. Storing the Data (minDB anyone?)
  3. Viewing this Data in HTML elements. Bascially an easy task, this could make YASP a (mainly) JAVA applet aswell. The desktop-app could easily feature the full server-set with a custom client app ontop. This has the big advantage that features of the server could be easily deployed in the client. The biggest issue will be the database: A FileDB for the desktop, a real DB for the server. Might need only a planned queue, as querying data is basically the same.

Oh, one thing to note: Threading the SteamAPI and inserting the MatchData on filebased DBs will horribly fail without queue-systems. I've experienced it quite a lot, as writing in it takes ages (up to 0.25s) and errors out the match-query.

2.) Inserting into the DB: You can connect secure db-interfaces with public keys. Depending on the algorithm, you can bind them to your Java Build and create a public key peer build and validate it on the server with a private key. It's pretty secure and is normally not so hackable. From there on people can parse matchIDs, throw them in the DB and your parsing-server doesn't need to parse when people do it the same. As Workload is getting an increasing problem, you could give people the option to parse their matches on their desktop app or can buy a "premium"-service, which parses the matches on your server. This also opens up a lot of resources for a proper API, which could people allow a match-synchro when they're deploying clients on new systems. Offload for everyone, boyz!

3.) On a side note: Maybe the idea of getting away from Java for a client-application wouldn't be a bad idea, but would be harder to deploy on different system (I have no experience with multi-system programming in C++, for example). That said, Java shouldn't be the biggest issue, since we're not facing any time-critical issues.

howardchung commented 9 years ago

Won't the public key in the client be accessible if the program is decompiled?

DarkMio commented 9 years ago

Ofc it will, but that's why we only give out public keys. Connect with a secured connection, verify it is a genuine client and let them insert. A buffered database connection would be nice, as it will prevent DDOSing the database by spamming it. Threaded by users (a bit memory hungry in theory) it could close connections. Maybe it will autoflagged as user-import, since you can't be always sure that someone is inserting with a genuine dataset.

Edit: Heh, you could even write a native piece of code + obfuscator for the data connection, making it way harder to decompile and even connect to the buffer.

Editedit: I have thought about the issue now for a while around. The issue here is a conflict of interest. If you say that the websites scope is a better dotabuff, then you will run into issues when people inject their faulty stats (because "I am the King of Dota" or something). If the scope is more about personal tracking, then let them chunk around with data. Data-Validation could be pretty easy, as you can compare the endstats with what people insert into the database AND you can validate it with what people parsed on their side (if two people use the same service in the same match). If it is any different, you can either reject it (or even ban people after multiple abuse) or just ignore it. You won't be able to trust the client in any way, yet you can offer it as a private extension and offloading your parsing process.

pyarmak commented 9 years ago

I believe what you guys are looking for is sort of like my application is it not? https://github.com/d2rm/d2rm My application is cross platform and is based on node-webkit which would allow the code base to remain close to the website's and make it less of a pain to maintain. It would be fairly trivial to add uploading functionality to the application (including private matches and MMR history).

I began the application based on your code (but it has diverged since then because I'm the only developer) so perhaps that's something you guys could look into. The code needs some cleanup but it wouldn't be too hard. I'd love to collaborate with you guys seeing as it's rather difficult for me to keep up with developing this application all on my own.

howardchung commented 9 years ago

Closing as we have no plans to develop this.

@pyarmak We've made some pretty significant changes to the YASP parser output format. If you like, you can update d2rm to support this new format, but I don't think we have the manpower to devote development time to keep both YASP and d2rm compatible with the parser output. You'll have to be in charge of keeping d2rm up to date if you want to use the latest parser upstream.

pyarmak commented 9 years ago

@howardc93 Thanks for the heads up. I am currently away from home for medical school interviews so I haven't had time to work on d2rm but I will pull the updates from YASP after March 14th when I get back.