search

koraktor / steam-condenser

A multi-language library for querying the Steam Community, Source, GoldSrc servers and Steam master servers
https://koraktor.de/steam-condenser
Other
359 stars 65 forks source link

Provide support for ISteamUserStats API #157

Open koraktor opened 12 years ago

koraktor commented 12 years ago

This seems to be a bit WIP on Valve's end too, but it might be a better alternative to GameStats (and subclasses) in the future.

It could provide a general and localizable interface to all available game statistics.

koraktor commented 12 years ago

Moving this to 2.0.0.

I don't want to replace GameStats right now and enforce users to have a valid Web API key. But expect this to happen soon after 1.3.0 is released.

Yeggstry commented 11 years ago

I would be willing to help with the Java implementation of this feature. I am currently looking at implementing something similar in my own code, since there are some games (e.g. Tower Wars) that do not have a valid xml feed.

It is worth noting that the data supplied in the Web API is a subset of the information supplied in the xml feed. For instance, only the steam ID and game name are provided, as well as a list of the achievements' apiname and whether it has been achieved.

koraktor commented 11 years ago

Are you familiar with Ruby? I have an early WIP branch with a few classes for this purpose. If you want, I'll publish this so you can have a look at how this could be done.

Yeggstry commented 11 years ago

I'm not familiar with Ruby, but I'm a quick study :)

koraktor commented 11 years ago

Here we go: wip/user-stats-api It should work to a certain degree.

Yeggstry commented 11 years ago

Thanks, I'll take a look.

Yeggstry commented 11 years ago

Well I've spent about an hour scrabbling around to understand what was going on (mostly reading Ruby for the first time :)), then a lightbulb came on!

I didn't realise that there was a GetSchemaForGame API call, since the Valve Wiki doesn't mention it: https://developer.valvesoftware.com/wiki/Steam_Web_API. If you have a complete list of the Web API calls for ISteamUserStats could you supply them here please, I've found a few but the documentation around seems to be incomplete. I guess the main ones are GetSchemaForGame and GetUserStatsForGame.

Is it the intention that the steam-condenser will allow the ability to use either the "old" xml feed and the Web API? I think part of the reason I was confused at first was that there are ruby files which have very similar names but use different methods to retrieve their data i.e. game_stats uses the xml feed, game_stats_schema and game_stats_value uses the Web API.

Sorry for all the questions, its useful to know what approach you are taking before I dive in.

koraktor commented 11 years ago

There's ISteamWebAPIUtil/GetSupportedAPIList/v0001 for a full list of API methods (without an API key it will list only "open" methods that don't need a key).

I'm not entirely sure if I'm going to drop the old XML-based GameStats (and similar classes) right with 2.0.0. It depends on how the current functionality maps onto the Web API.

To better see what's new in that WIP branch have a look at koraktor/steam-condenser-ruby@39e581c94f0ee5b1b946c26af020ea5dcaee0e1c.

Yeggstry commented 11 years ago

Well I'm working through this at a steady rate at the moment. Doesn't help that the API is quite.... quirky :) I'll explain more once I've got through it, but I've got some notes that would be worth sharing.

At the moment everything I've done is completely independent of the Java code you've written, I don't want to distract the code I'm writing with trying to merge them in for now.

I also took the decision to create a service class for the ISteamUserStats. This should make it easier to see which services aren't implemented and what versions are being used of each service.

Hopefully over the weekend I can check my initial work in for review :)

Yeggstry commented 11 years ago

Ok I have checked in my code for the following ISteamUserStats services:

My user stats api branch can be found here: https://github.com/Yeggstry/steam-condenser-java/tree/wip/user-stats-api

I am still getting used to git so I think that some of the changes I have committed are changes I shouldn't have (such as the version number in the pom). My unit tests do also depend on the WebApiException fix you've put in, but I haven't figured out how to do that yet :)

I would appreciate a review of the changes I've made, I've only touched a couple of the currently existing files (SteamGame by mistake, and WebApi in order to get a top-level JSONObject).

I still need to do the following:

Any comments you may have are welcome, this is the first time I've worked on an API so there may be some things that I haven't done that are best practice.

Yeggstry commented 11 years ago

Oh, and I am aware that the GetSchemaForGame method is calling the wrong service, i was just having problems with committing that change :)

Yeggstry commented 11 years ago

I am planning on updating the unit tests I created so that it covers the problem I mentioned in the previous comment, as I am currently using a generic call to WebApi.getJSONObject i.e.

when(WebApi.getJSONObject(Mockito.anyString(), Mockito.anyString(), Mockito.anyInt(), Mockito.anyMapOf(String.class, Object.class))).thenReturn(userStatsForGameDocument);

However, when I tried to do something similar to the code in SteamIdTest.resolveVanityUrlSuccess(), it doesn't match the when since the params Object in the test is a different object to the params Object created in the SteamId.resolveVanityUrl. Can you provide any insight to resolve this please?

@Test
public void testResolveVanityUrlSuccess() throws Exception {
    HashMap<String, Object> params = new HashMap<String, Object>();
    params.put("vanityurl", "koraktor");

    mockStatic(WebApi.class);
    when(WebApi.getJSON("ISteamUser", "ResolveVanityURL", 1, params)).
        thenReturn("{ \"response\": { \"success\": 1, \"steamid\": \"76561197961384956\" } }");

    Long steamID64 = SteamId.resolveVanityUrl("koraktor");
    assertThat(steamID64, is(equalTo(76561197961384956L)));
}
Yeggstry commented 11 years ago

Right, I completely screwed up the old branch (don't ask, I'm new to this :)), so I deleted it and created branch user_stats_api. I also fixed getSchemaForGame calling the wrong operation.

I figured out what I was doing wrong with trying to match the HashMaps - it was trying to do an equals on Long and Integer, which it clearly didn't like. Converted them all to String and it worked :)

I still need to do the following:

Hopefully this should be in a usable state now for comments.

I am on holiday from tomorrow so I'll document my changes when I come back.

Yeggstry commented 11 years ago

I've also found an issue with the response of some GetUserStatsForGame calls, and have mentioned it in the Steam forums. Not sure its the right place for it but couldn't find anywhere else :)

TheSeg commented 11 years ago

So looking at how stats are handled now and the move to WebAPI, I have a proposal to make.

Standardize all data collection of the user stats.

Using methods: GetUserStatsForGame, GetPlayerAchievements & IEconItems_*

Right now stats are a custom job. Apps with stats are treated with custom code, replicated per platform. This is a product of the XML system which warranted this approach. WebAPI does a good job with standardizing the information by standardizing the request under the GetUserStatsForGame method. So I ask we pass the stats over wholesale. That way we support every game past present and future.

GetPlayerAchievements will also fill the job with achievements. IEconItems_* method(s) will deal with items. I confess I'm not as researched on items yet, but the docs seem to be similar across games; Only the * replaced with the AppID.

Allow custom data on top of WebAPI's stats info.

While we have the above, there are some games which can benefit from post-processing of the stats data to derive analyzed data. Calculations based on stats and processed outside of WebAPI.

Taking Puzzle Agent 2 for example, the Taxpayer dollars figure isn't provided by the Stats API. Instead, the com.telltalegames.grickle102.totalSubmissions field is multiplied by $97,864.10.

Steam Condenser can provide the added values on top of the existing data structure when developed, but still grow as titles expand without extra development.

What changes?

Right now stats are a switch statement which either pulls a default call ''OR'' a fully custom call. Instead, we should expose the WebAPI's stats natively for all games. Then optionally adding custom functions on top where developed.

Yeggstry commented 11 years ago

This is very similar to what I was thinking when I did the work on the WebAPI for the Java code.

For instance, I would expect a game-specific class which would extend UserStats that could not only provide user-friendly get methods (similar to the way that the current TF2 API), but also derived values as you have specified above.

Since I didn't know the main code for this project at the time, I wanted to get the plumbing in place (i.e. the WebAPI calls) before updating the existing API. I think there are two more I wanted to do before I was confident that the XML code could be removed.

Its worth pointing out that GetUserStatsForGame provides the closed achievements for a game, and that GetPlayerAchievements provides all achivements. I can see situations where only one set of this information is required, I'm not sure if both would be necessary.

TheSeg commented 11 years ago

That's right! I wasn't thinking about achievements and forgot GetUserStatsForGame already provides that info. Might as well use that one call for all data -- no need to nab GetPlayerAcheivements again.

Keep in mind that we can't completely remove XML. There's currently no way to get Group information from WebAPI beyond a player's subscriptions. Everything else seems to work OK though.

Yeggstry commented 11 years ago

Ah yes, forgot about the groups. Looks like this has already been requested, but it might be worth giving them a polite nudge :)

One thing that I forgot to mention when I wrote the code for this task is that there is no "unlockTimestamp" for closed achievements in the WebAPI. I know that you'll probably do a similar thing with the games list as you have with the profile and you'll spot this, but thought I would highlight it. It is a useful feature to have, in fact I use it on my gaming community website. Half of the pages would be useless without it :(

koraktor commented 11 years ago

:+1:

Having one main stats implementation is mandatory. The current way already tries to do this, but that only works for some very basic attributes and the achievements of a game. The XML API is really lousy in this regard. My fear is that this might not be possible for all games, Valve continuously fails to provide consistent APIs, that's why inventories already need quite some customization.

But like @TheSeg mentions, it should be no problem to provide custom implementations for single games that build on top of the basic stats classes. I really hope that no additional customization is necessary, e.g. new API calls, attributes etc.

@Yeggstry I constantly try to nudge Valve about the Web API. Current status: No response at all.

TheSeg commented 11 years ago

I'll start working on the PHP side of Stats & Achievements for users. I may also add a new call for Game Scheme, but independent for pulling Stats. May not start today, but will have something by end of next week.

@koraktor I'm also nudging my Valve contacts. No progress so far either, but will keep soldiering on.