Parse HotS s2gs

[dsjoerg]

summary = sc2reader.load_game_summary("http://xx.depot.battle.net:1119/0aab1dfe9d4a608527e279fe45ff8f503421cf486df869ee25ce2bbae3cb3ad1.s2gs") print summary.build_orders[0][0] BuildEntry(supply=1, total_supply=0, time=0, order='Units Trained', build_index=0)

[dsjoerg]

I played this match, and I random'd Zerg (meh) and the first thing I built was a Drone.

[dsjoerg]

This s2gs is from the latest patch

[dsjoerg]

Interesting, the SC2 beta client itself has trouble parsing this s2gs. The graph screens give an error within the SC2 client. Here are the screens it is able to show: http://imgur.com/a/OTezq

[dsjoerg]

Here's a fresh one that is having problems.

summary = sc2reader.load_game_summary("http://xx.depot.battle.net:1119/8f677049dbe3821b51fe15dafd91dff89056dbca370b9a8f8d9945ead60270b8.s2gs")
summary.player[0].build_order[0]
>> BuildEntry(supply=17, total_supply=0, time=0, order='Structures Razed Count', build_index=0)

[dsjoerg]

Here's an s2gs I played in together with Score Summary images: http://xx.depot.battle.net:1119/8f677049dbe3821b51fe15dafd91dff89056dbca370b9a8f8d9945ead60270b8.s2gs http://imgur.com/a/tKQZQ

[dsjoerg]

Another: http://xx.depot.battle.net:1119/3fb7884473a7ae1972d605f2581435ded4d63938667041cebe7735c157708769.s2gs http://imgur.com/a/vzup1

[dsjoerg]

And one more: http://xx.depot.battle.net:1119/4ab77afac6dbdf7e2fe542db2350772a4c71dd245d610ac304ff9e5964be0b98.s2gs http://imgur.com/a/7hEqC

[GraylinKim]

Imagine the end s2gs file as a giant array of statistics, all with the exact same serialized form. They are ordered by type: summary stats -> graphs -> build order. The issues we are seeing here are because there are now 3 additional summary stats and 2 additional graphs which shift the rest of the entries back.

We've been using fixed indices to identify the start and end of one chunk of stats. I'm going to look and see if there is a way I can code with without fixing the indices. It would be much easier if we were able to identify the version of the file but it isn't clear how we would do that.

@dsjoerg, could you get me an archive of WoL and HotS files? I might be able to find a version identifier by comparing the two.

[dsjoerg]

Here are 1000 WoL s2gs hashes: http://pastebin.com/v00V120J they were retrieved today.

I'll get you some HotS hashes later today. They may be trickier to work with because there may be some older formats mixed in there that nobody cares about any more.

[dsjoerg]

Actually you need gateway and hash in order to work with these. http://pastebin.com/KrswxbtW

[dsjoerg]

Here's some trivial code I wrote for playing with HotS hashes that you can adapt for WoL

import sc2reader

hots_hashes = open('/tmp/hotshash.txt','rb')
for hots_hash in hots_hashes:
    url = "http://xx.depot.battle.net:1119/{}.s2gs".format(hots_hash.strip())
    print "Getting {}".format(url)
    summary = sc2reader.load_game_summary(url)

[dsjoerg]

Sorry for being captain obvious

[dsjoerg]

Here are 245 HotS hashes: http://pastebin.com/CQrawqFE all retrievable from gateway xx of course.

If you need more it's about 5 minutes work to get this many more hashes.

[GraylinKim]

Thanks @dsjoerg, this should be sufficient.

[GraylinKim]

This is fixed by ac1e114a4c3507fb1dfed52feb7ca04bd2429d5f. Holding the issue open for verification (by me).

[GraylinKim]

Have you already verified this @dsjoerg or were you just looking a successful parsing and not the details.

[dsjoerg]

I have not verified this. I know that the code can parse a HotS s2gs without failing, but that's all I know.

[dsjoerg]

I've confirmed that buildorder parsing is failing for at least one HotS and one WoL s2gs. It's getting values like 'Workers Active' and 'Army Value' for the order rather than a legit order such as 'Drone'.

Added a test. Debugging now.

[GraylinKim]

The build order issue is fixed (with test added) in 71638b01e06e0e1097f6774b03fabfb526cacbdf.

[GraylinKim]

Okay, all the s2gs work has been merged back into HotS on the main repo. Time to close this.