Pulling the chesses/level/tier data from a server

[boingy]

One thing I was thinking was it could be better to have all the data (chesses, PvE rounds, translations, probabilities, pool size etc.) available via an API then at the start of a game have the UI make a http request for them and then store them in objects to be used by the UI add-on normally.

The advantage of this approach would be that when they make an update such as adding/removing heroes or we want to update the tier list, we could do the update server side and then people wouldn't have to update their VPK. This would be especially useful because at least for me I haven't had to change my gameinfo.gi file despite several Dota 2 updates so if done right a user wouldn't have to ever update.

The challenge would be doing it in such a way that we make as few assumptions as possible in the UI code. For example if we did this before IO was released chances are we would not have accounted for the fact that a 5-cost chess piece could have equal probability at every courier level and have infinite copies of itself in the pool. So all that data (copies of itself in the pool, probability at each level) would need to come from the server.

There's also all sorts of other changes we would need to flexible around:

• PvE round numbers changing completely
• A unit that becomes tier 2/tier 3 by combining a number of copies of itself other than 3
• Number of couriers levels are changed
• Another cost of units is added or removed (would be tricky with the way I made that UI) and more...

I can take a look at this and try and mock up a schema for the data that would work for as many eventualities as possible.

Just let me know if you have any objections to this approach.

[TiesdeKok]

I have actually been playing with this idea as well, the reason I haven't done it yet is that this might add a layer of upkeep (i.e. if the API goes down people can't use it anymore). However, I think it would be good direction overall. A couple of suggestions:

1) I suggest to use Firebase as it is free and it scales nicely, also, it has good up time.

2) Ideally we would like to have a fallback solution for when the API is unavailable. In a perfect world this would be a cached version of the most recently downloaded data, however, I don't know whether it is possible to create a cached version (i.e. write to the file system) from within the Panorama version of Javascript. If this would be possible, we could also implement a version check to decrease the amount of data that needs to be downloaded (i.e. if the version of the cached data equals the live version, then don't download, otherwise update).

3) It would be nice to have a file schema that is flexible, but I don't think that we can completely avoid VPK updates. This approach would definitely reduce the need for VPK updates, but for major overhauls it is unavoidable I think. But, if we have an API connection set up we could definitely include a flag in the DB that we can change on the fly to show or hide a message that there is an updated VPK available for download. This would go a long way, in particular if it doesn't happen frequently (as most of the updates would go "over the air").

4) It would be very cool to have a feature (maybe a Github page?) where people could link their steam account and then select their own tier list. We could then store this data in the Firebase database linked to their steam ID, to give people their own tier indicators in game. This is definitely feasible I think, it would just take a little bit of work of creating such a web-app. We could maybe modify the other tier list creator that is already hosted on Github (https://rautochess.github.io).

Either way, good idea!

[boingy]

1. In general I don't think upkeep/hosting should be a problem. We are just going to have a static .json file, there's no need to be pulling stuff from a database or anything since every request is going to get the same result (with a possible exception around languages, would probably want a different endpoint for chinese for instance rather than sending chinese strings to everybody). I envision a CRUD type page that allows us to update a DB or something that then modifies that static json file, but when a request is made for the data from the UI to the server the DB won't be touched. There are hosting solutions that would make this essentially free even with many thousands of requests

2. Fallback: If we keep the hosting simple (like hosting a static .json file on S3 or firebase or something) then I don't think we should be concerned about downtime. We could have a backup 2nd URL hosted for redundancy that gets requested if the first 1 fails but I don't envision this being a problem. Caching: yeah I don't see a way to write or cache anything in the panorama API although the docs aren't great so it's possible I'm missing it. I need to look into how the UI add-on is loaded (it looks like a message is logged into the console before the autochess custom game is ever launched) but it might be possible to store the data in memory that would persist as long as the game is opened so at least we aren't downloading it every single match in a single play session. Again though I don't think it is a big deal if a static .json file is requested once every half hour by every person that has the mod. I don't know if you have insight into how many people are using the mod but I can't see this needing to cost anything more than single digit dollars a month if even that, a lot of these cloud providers have free monthly allowances which would be ideal for this kind of use case.

3. Agreed, I think in particular the kinds of places where a VPK update will be unavoidable would be if they were to change the layout of the mod meaning your probability labels on the draw screen would be in the wrong place or if they changed the way data is sent in the callbacks we rely on. I do think though as long as we are flexible and make the UI code as 'dumb' or agnostic as possible we could get away with not requiring VPK updates very much at all. In the past 4 months despite quite a lot of updates the only exceptional thing as far as I can remember has been the way IO is handled differently in the pool.

4. I could definitely see that being a good feature when I watch twitch streams and a streamer has a tierlist a lot of people want to see it. I guess the idea would be to attach a tierlist to a user's steamId once they have used the openid signin then make a request from the UI including their steamid as a param. I think as well as letting people make their own lists it would be good to let them select shared ones from streamers or other community members similar to how there are websites where people share hearthstone decks. I think that could be a compelling feature for people to download the mod in the first place rather than having to alt tab a bunch.

[TiesdeKok]

You are right, without the personalized tier list functionality it boils down to read-only, hadn't realized that. Hosting a JSON file in a redundant way is indeed easy enough and wouldn't require anything fancy. We could probably even just host it via Github pages, the up-time of that should be good enough (the download repo gets between 20 to 60 unique users a day so the numbers shouldn't pose any problem).

With regards to 2), the mod is loaded once when you start Dota 2 and hooks into the events used by the Auto Chess mod. So by construction this would imply that the API call is only made once per session until the game is restarted.

[boingy]

so I finally had a chance to look at this. I got a model for the chesses and game rules that I think would be fairly robust to most changes and I was in the process of changing some of the existing code to accommodate it but then I got thinking about how unsatisfactory it is that changes to the UI like in the last patch or if they were to change the format of the data in the battle_info etc. callbacks it could still mean we will have to do an update.

On a whim I tried this:

(function () {
    $.AsyncWebRequest('https://raw.githubusercontent.com/auto-chess-ui-mod/source/master/dota_hud_pregame.js', {
        type: 'GET',
        complete: function (res) {
            var data = res.responseText;
            data = data.substr(0, data.length -1);  
            eval(data);
        }
    })
})();

and it works!

That's it, that's the whole VPK. So now anytime an update is pushed to the repo users will get it. Would need to add a little bit to remove the probability/tier code blocks between the /*START-DRAWSTAT*/ strings etc. depending on user preference

[TiesdeKok]

Of course, that is a really clever way of doing it! Can't believe I didn't think of that earlier.

The easiest way to still have different version would probably be to just create separate source files.

(I am in the middle of moving from Europe to the US right now, hence me not being so pro-active in development myself currently)