search

sc2-arcade-watcher / issue-tracker

1 stars 0 forks source link

Match history tracking #6

Open Talv opened 3 years ago

Talv commented 3 years ago

Since ~2020-12-01 sc2arcade.com has been actively tracking match history of individual players. Now, since beginning of this month it can track outcomes of public lobbies using the same data.

It was made to work in real-time, but it hasn't been refined yet, and the allocated resources for this purpose are artificially constrained.. so It means that depending on the map/lobby/time of day etc. the delay between when the match actually ends and when AW registers that fact, can range between 1m to an hour or more.

How it works

It uses Blizzard's WebAPI to query match history of individual profiles (example: https://starcraft2.com/en-us/api/sc2/legacy/profile/2/1/2642502/matches).

Unfortunately, the data given out by this endpoint is rather sparse.. what we can get about individual match is this:

        {
            "date": 1611224981,
            "decision": "Loss",
            "map": "Wall In, Wall Out!",
            "speed": "Faster",
            "type": "Custom"
        }

Limitations on Blizzard's end:

Now, when we have a mapId and timestamp we simply find other players with the same set of data. And map that information onto applicable public lobby where players have been seen together.

Can privately hosted lobbies be tracked?

No. At least not reliably, since system has no data about private lobby to begin with. Thus the only thing we got to work with is timestamp - this isn't enough. Although some guessing may be performed on that data, and website in near future will provide some features to facilitate this purpose. However it'll be on-demand kind of thing.

Error codes

Whenever a match for lobby cannot be found, a lobby will be flagged with non zero result: image (Notice: E(3))

export enum S2LobbyMatchResult {
    Success = 0,
    MapInfoMissing = 1,
    SlotsDataMissing = 2,
    HumanSlotCountMissmatch = 3,
    HumanSlotProfileMissing = 4,
    HumanSlotDataCorrupted = 5,
    UncertainTimestampAdditionalMatches = 6,
    DidNotStart = 7,
    UncertainTimestampPlayerDuplicates = 8,
    Unknown = 255,
}

HumanSlotCountMissmatch - can often be seen in maps which use low lobby startup delay, such as 3s/5s.. It means there's missmatch between last stored lobby snapshot (the slotsTaken/slotsTotal that can be seen on open lobbies list, in game) between last retrieved lobby preview (it includes complete & non-cached list of players in particular lobby).

In most of the cases it simply means that lobby has been started, before the bot could retrieve up to date lobby preview. Map authors can solve that problem by setting lobby delay to at least 10s for every game variant. And it's recommended thing to do, for anyone interested in whatever sc2arcade.com offers. Since that's not the only system affected by this limitation.

Unknown - couldn't find a matching set for lobby, due to unclassified reason. It timed out after 9h.

As for the rest, refer to source code: https://github.com/sc2-arcade-watcher/server/blob/92a06c04f925cc6c4de3ebd3daadc0871b0398b1/src/bnet/battleTracker.ts

Discord bot integration

Soon..

Plans:

Ideas:

ELO / Rating / Leader boards

With this system in place, it's now possible to build server-side rankings for any map that uses standard win/loss conditions. Team based, or not.

I have no immediate plans to work on that. However, since it could be made standalone, mostly separate from the rest of AW architecture, it's something that could be developed individually (and further integrated at a later date). It would be an excellent contribution to the project if anyone is up for the challenge.

The sample data needed for this system to work can be obtained via API. See below.

API

https://api.sc2arcade.com/lobbies/history?regionId=2&mapId=140436&includeSlots=true&includeSlotsProfile=true&includeMatchResult=true&includeMatchPlayers=true&orderDirection=asc&limit=100&after=WzE4OTU1NDY3XQ== (Check the docs for additional information regarding pagination etc.)

Notice: Only lobbies after 2021-02-01 00:00 (UTC) will have non null match property. However as the system was collecting match history actively since 2020-12-01 it's possible to retroactively index lobbies to that date. It's planned.

ipax77 commented 1 year ago

Very cool stuff, thank you! While gathering the data for Direct Strike ratings I recognized ~25% of the replay-data do have at least one player profile with ProfileId = 0.

Would it be possible to fix this?

lobbies/history?regionId=1&mapId=208271&profileHandle=PAX&orderDirection=desc&includeMapInfo=true&includeSlots=true&includeMatchResult=true&includeMatchPlayers=true

image

Talv commented 1 year ago

I think 25% is a stretch.. unless you include all kinds of failures regarding matching lobby info against the match history records.

Because throughout the years there were many issues on my end as well as Blizzard's end (there were at least two periods where Blizzrard's webapi was dead for like few months).. So they may be gaps in the history (where it only recorded lobby info, but no match results). But since like ~2022-10 stuff was running pretty stable.

But anyway, what you should look like to determine if the match result is valid is the match.result property returned in various endpoints.

Such as here: https://sc2arcade.com/lobby/1/1661880554/39762588

image

image

This one has Failed to identify with following error code: HumanSlotCountMissmatch = 3, From what I can see, it's because the host kicked 5th person and then proceed to start the lobby immediately making it 2v2 (or that person left), and the bot responsible for monitoring lobbies lagged/was overloaded/was performing scheduled restart|relogin/whatever-else and missed that event. And as such match outcome from this lobby has failed to be identified.

Since you've not given me the exact lobby id, I'm not sure about which record you're talking about. But it is indeed possible that sometimes players within the lobby won't be identified. And there's only name... in such case the those lobbies are immediately flagged with result of HumanSlotProfileMissing = 4, (see: https://github.com/sc2-arcade-watcher/server/blob/1f333ca03ce5be5f3f57f155aceec86189a974ed/src/bnet/battleTracker.ts#L591-L594 and https://github.com/sc2-arcade-watcher/server/blob/1f333ca03ce5be5f3f57f155aceec86189a974ed/src/entity/S2LobbyMatch.ts#L6-L18)

This is related to a memory corruption related bug within the bot that monitors the lobbies (it's not open sourced). I've been impoving it over the years, to the point where I believe it'll miss at most like 50 records a day (from all lobbies across all region on arcade - not just DS).. and also during other days have a streak when it doesn't miss anything. And while I have a final fix for that which will eliminate issue altogether, it's part of the next version. Which I'm slowly working on.. it might be see the light by the end of this year, as a part of larger infrastructure upgrade.

But yeah, overall you unfortunately can't expect that all arcade lobbies will have match result associated.. Especially the further back you go in the history. When it comes to last ~6 months, it should be at ~95% easily, I think. At least in case of DS.

ipax77 commented 1 year ago

Thank you for your quick reply! What else than the Id in the screenshot you would need to identify the lobby? My data is filtered for 3v3 lobbies only source code

            if (result.SlotsHumansTaken != 6
                || result.Match == null
                || result.Match.ProfileMatches.Count == 0)
            {
                continue;
            }

Any plans on making the monitor bot open source so people (me) can help out?

Talv commented 1 year ago

What else than the Id in the screenshot you would need to identify the lobby?

I'm not sure to which entity/record those IDs correspond. Are these the IDs you're getting from the API, or something internal on your side? I'm speaking about this:

image

If they're corresponding to the lobbyId on my side, then why does the second column show a player name? There can be multiple players in the lobby. I suppose I'm only seeing small part of your DB setup, but anyway.. let's take the first ID 77602744, it corresponds to: https://sc2arcade.com/lobby/1/1661880554/39290420. And by looking at the participiants, I don't see anyone with the name of BoonToon.

Furthermore I advise to NOT relay on these IDs return by lobby/history endpoint. I never meant to expose them in the first place, there were meant for internal use, and as noted in the docs, they'll likely be removed in v2 of the API (https://github.com/sc2-arcade-watcher/server/blob/1f333ca03ce5be5f3f57f155aceec86189a974ed/src/api/routes/lobby/history.ts#L29).

Unique lobby should be described by a combination of {regionId},{bnetBucketId},{bnetRecordId}. Those are the IDs I use in the URLs like the one linked above for instance. And they are guaranteed NOT to change, because they directly correspond to IDs returned by Battle.net (Blizzard internally calls these AdvertHandle).

            if (result.SlotsHumansTaken != 6
                || result.Match == null
                || result.Match.ProfileMatches.Count == 0)
            {
                continue;
            }

Yeah, this looks okay. Although something like result.Match?.Result != 0 would probably cover all scenarios. Because:

Also, when relaying on names, remember that players are able to change character names at any time. So, when you dig in the lobbies from the way back, then the names in the slots section will return the names of players they had at this point in time. However, their IDs (realm & profileId) will always be the same. So a general advice - try to avoid matching records by name.

Any plans on making the monitor bot open source so people (me) can help out?

That's a tricky matter, because at the moment the program that monitor lobbies has a lot of dependencies.. and is highly coupled with a SC2 map hack code, that was shared to me so that I could have the easier time bypassing some of SC2 anti tampering shit. I can't share it as a whole.

That said, I have actually started refactoring it slowly some time ago, and once I isolate the bare minimum code required for monitoring lobbies, and get rid of everything else that I'd not like to share (SC2 map hack specific stuff), it's likely that I'll share it with interesting parties, and perhaps eventually open source. But there's a long road until that happens. I'd not expect it sooner than around Q4 this year (but even that is mostly a guess, because I don't have a specific timeline as to what I plan to do and when exactly).

Talv commented 1 year ago

let's take the first ID 77602744, it corresponds to: https://sc2arcade.com/lobby/1/1661880554/39290420. And by looking at the participiants, I don't see anyone with the name of BoonToon.

Actually.. wait, I do see it:

image

image

It is because of what I've mentioned above:

Also, when relaying on names, remember that players are able to change character names at any time. So, when you dig in the lobbies from the way back, then the names in the slots section will return the names of players they had at this point in time. However, their IDs (realm & profileId) will always be the same. So a general advice - try to avoid matching records by name.