Parsers

[jaagupkymmel]

When writing parsers for non-production purposes, make sure you make them as strict as possible, to make it easier to find errors in wiki.

[lukegb]

:+1:

[themasch]

I totally agree with that. lol-observer is way to generous. Needs to be fixed :D

[tyscorp]

Perhaps we should write a "reference parser" and include unit tests for everything, then run it through all featured games for every region.

[themasch]

@tyscorp i really like this idea. We should use a language everyone understand and thats easy to work with and easy to depoly. Ideas? I'd vote for js..

[tyscorp]

I'd vote for node ;)

[Zero3]

I plan on expanding https://github.com/Zero3/LolSpecAnalyzer to include a proper parser once I get some free time. It's Java.

[jaagupkymmel]

The one problem that I have with JS is dynamic typing. It's harder to figure out of which type a parameter for a function should be. If you're willing to add comments about params and return types, or switch to something like http://www.typescriptlang.org/ (Never used, just Googled), which is essentially JS, but has static typing, then I guess I'll be fine with JS, even though I'd prefer Java.

@Zero3, if you don't feel like writing a new parser from scratch, you may contribute to https://github.com/loldevs/lol-spec, it does fail some keyframes, but this is intentional, as it means everything that does not correlate with the information in wiki should raise an exception and thus making it easier to find edge-cases/errors in wiki. Right now, it needs to collect information about sections we know nothing about, and maybe analyse this data.

[themasch]

@jaagupkymmel I understand your point even when I'm a pretty big fan of dynamic typing. But I'm 100% with you regarding comments, if we write one "standard loldev leaguespec parser" there need to be a lot of comments everywhere!

I'm not really sure about typescript because its one of these "compile to js" languages. Developing with them always feels awkward.

Personally I'd prefer C++ over Java if we choose to not use JS but I'm fine with anything thats a "common used language". So no Haskell/COBOL/Basic/brainfuck pls. If you bring really good weapons I might even be okay with .NET. Maybe. If I'm drunk...

[Divi]

I'm clearly not good with data sniffing, but Java is often used for these uses. Fast to develop, and good objects typing. I'm ok with JS too.

[jaagupkymmel]

C++ seems a bit of an overkill for such task. Looking at the codebase we have right now, we should stick to either JS or Java.

[themasch]

Okay, should we put the parser inside this repo (since the documentation is in the wiki anyway) or should we create a new one?

[jaagupkymmel]

I think we should have a seperate one.

[vafrederico]

I think we should put it in this one, since the parser will be directly related to the wiki.

By the way, i have a parser written in Python. In case anyone need it.

[themasch]

Maybe we could just prepare some structured language-agnostic test data and provide multiple implementations ins various languages. Every one of them tested against the same set of tests.

[toulouse]

More importantly, can we have a test set of data to write parsers against? I'm interested in writing a parser, myself, but can't find *.rofl files, and the PBE seems to be broken and disallowing replay downloads.

EDIT: Er. What @themasch said.

[vafrederico]

I have three replays saved, already decrypted. And i could download a few more. Where should I commit them? In this repo, or we will have another repo for parsers?

[Divi]

I don't think that commit decrypted file is a good idea. But you can add links to pastebin or gist to your decrypted files in the wiki.

[themasch]

I think we should create a parser-spec repo & collect some inputs & expected outputs decoupled from full files. Does someone know a good format for language agnostic tests? I'd suggest base64 encoded input & json output but we need to agree over a meta format.

[Divi]

Json is understood by all languages (with libs or not) and can be easily read by developpers : :+1:

[Zero3]

Regarding my comment @ 01-03-2014: Exams are almost over, so I've started my work on a proper parser. I've piggybacked on the downloader by @jaagupkymmel (https://github.com/loldevs/lol-spec) to fetch and decrypt all keyframes for a game which worked great. So I have a test game to work with now.

My plan so far is to import keyframes from disk, parse them into Java objects and output the data in JSON format, keeping the framework modular so that one may easily add additional importers and exporters.

If you guys have any preferences/ideas/notes before I get too far, be sure to let me know! I agree about the strict parsing comment, which I will keep in mind. And the JSON comment as well, of course.

[Divi]

Good luck for the end of your exams :) Great idea about output formatter. For the input, maybe you should create an input which parse a keyframe directly from an URL, without downloading the keyframe (on the disk), for people who don't want to keep the keyframe.

[themasch]

Do you guys think its a good idea to agree on a standard output format for keyframes and chunk? Like a JSON schema?

[jaagupkymmel]

Yeah, JSON seems the way to go.

[akuseru]

I would recommend trying to match the /timeline endpoint shown in #12, while more data could be made available it would allow for some degree of switching out the official api with the custom data without too much effort.

[themasch]

That sounds reasonable. do we know if all informations from that timeline exist in key frames, too?

[Divi]

I think all data are in keyframes except the events part which are in chunks. There is no string like WARD_PLACED, etc, in keyframes.

The output should be the same as the timeline, with data from https://acs.leagueoflegends.com/v1/stats/game/NA1/1414415714, particularly the part ['participants'][n]['stats'].

I'll check this week-end if another data are available.

[akuseru]

@Divi events are available in the keyframes. Just because it's missing WARD_PLACED doesn't mean that the information is not there.

SightWard YellowTrinket do show up in the data.

[Divi]

I just took WARD_PLACED as example, I didn't find anything about kill or death. Maybe I miss something if you found a part talking about wards, I'll check this tonight.

[avborhanian]

Actually, I think the bottom section below turrets has more to deal with loading models and their placement for the game. Don't have a hundred percent proof, but Yonkey is listed twice as well as ShopMale, which are the models for the shopkeeper and his pet.