Possible changes to Destiny2.GetHistoricalStats

[dad2cl3]

I know the endpoint is still in beta so was curious if the following change might be coming to the endpoint. The current response from the endpoint for Iron Banner Control uses the key ironBannerControl as follows:

{"Response" : {"ironBannerControl": {stats}, "allPvE": {stats} ...

I can not find the key, as currently returned in the response, in the manifest. The corresponding key in the manifest is iron-banner-control. Fortunately, the integer mode in the Destiny.HistoricalStats.Definitions.DestinyActivityModeType is 43 and consistent with the manifest table.

A couple of other examples: pve (manifest) and allPvE (response), crucible (manifest) and allPvP (response)

So my question is whether or not work on that endpoint might include smoothing out the inconsistencies between the response from the endpoint and the manifest definitions? Possibly standardizing the keys? Utilizing the hash for the mode instead (consistent with aggregate activity stats)? Utilizing the integer game mode?

[vpzed]

Consistent naming/references would be a wonderful improvement. The current setup requires a lot of manual work to glue together what you get from one thing with what you get from other things (like the manifest - but also other endpoints).

[sgtfrankieboy]

The change you suggested to the GetHistoricalStats endpoint is breaking and should not happen.

The friendlyName property should only be used in urls. I think the best solution for this would be to add a new property to the DestinyActivityMode definition named modeTypeName that corresponds to the name of the modeType property.

[vthornheart-bng]

I would like to give this a pass at some point, perhaps in a v2 of the stats API. I don't know when I will have the time to do so, but indeed the API has a number of inconsistencies that have developed over the years that I'd like to smooth out. I'm going to put this on the enhancement wish list, but I should note that we'll probably have to deal with the sometimes-problematic-but-still-functional stats API as-is for the forseeable future. It'd be nice to at least readdress it for any theoretical sequel!

[joeydood]

I've used every single value for the enum, MembershipType and certain the membershipId is correct (as it pulls data when used in a different function.

Two questions: 1) How can we tell which endpoints are beta so as to not rely on them? 2) What is an alternative method for pulling stats?

Thanks.

[vthornheart-bng]

joeydood, what are you seeing when you make the call? What results are you getting that you weren't expecting? Can you provide the full URL of your request, and the data that you got in response? That will help us to determine what you're seeing that was unexpected.

All of the non-leaderboard Stats endpoints should be functioning, and are no longer in beta! In the documentation, the entries marked here as "Preview" are the ones not yet ready for widespread use:

https://bungie-net.github.io/multi/index.html

[joeydood]

Thank you for the quick reply.

It may just be that i'm a noob at the bungie api however, i've tried several combinations of MembershipType and Character values. This could easily be my misunderstanding of the api, my apologies if so. I've even tried the url without the Character parameter and values.

url: https://www.bungie.net/Platform/Destiny2/4/Account/{REDACTED}/Character/0/Stats/ This is the jsonconvert.deserializedobject value: { "ErrorCode": 217, "ThrottleSeconds": 0, "ErrorStatus": "UserCannotResolveCentralAccount", "Message": "We couldn't find the account you're looking for. The account may not exist, or we may be experiencing technical difficulties.", "MessageData": {} }

NOTE: The account value i am using is my bungie account number, it works fine in other functions such as: https://www.bungie.net/Platform/User/SearchUsers/?q=

[vthornheart-bng]

Ah, that's the problem - you unfortunately need to pass a character. We don't have a working endpoint at this time that returns stats across all characters on an account. Unfortunately, the error message it's returning isn't helpful. Can you share the Account ID you were using? (don't worry, account IDs aren't secret, it's safe to share that here) I can give it a test to see if that account is having problems in addition to not passing the character ID, or if that's the only issue.

[joeydood]

sure: 10281426

...looking for characterId enum...?

[vthornheart-bng]

Ah, okay - there's a couple things that you're running into then it sounds like! No worries though - this is unfortunately a very complicated part of the API, because of the way that we let a BNet account link multiple other accounts together.

The trick here is that 10281426 is your "Bungie.Net Membership ID". It's the membership ID that specifically represents this user as a Bungie.Net user, but it itself can't be used to look up Destiny information, at least not at this time. It's confusing enough that I'd like to provide a better way to do this in the future, but for now...

You'll know a membership ID is a BNet Membership Id because that is what gets returned if you're querying for user information in most UserServices queries. In other queries, if it's associated with MembershipType enum 254, that means it's the Bungie.Net Membership ID. (for instance, when you query SearchUsers, it returns an object of type "GeneralUser": those are Bungie.net users, which means their membership IDs are always Bungie.Net membership IDs)

What you're going to want do is use that BNet Membership ID to find out what Destiny Membership IDs are connected to it, and then you can pass those Membership IDs and MembershipTypes to the Stats endpoint.

You can get the other Membership IDs linked to your BNet Membership ID by calling:

/Platform/User/GetBungieAccount/{membershipId}/{membershipType}/

For the ID you gave, that'd be:

/Platform/User/GetBungieAccount/10281426/254/

(Well, shit... I just realized we're not returning info about this endpoint in the documentation! I will need to fix that, GetBungieAccount is very important for finding these linked accounts. Ugh! Sorry about that!)

In the results you get, you'll see a "destinyMemberships" property: a list of all other accounts that could have Destiny information. Each of them will have a membershipType and membershipId property: and those are the membership Types and IDs that you'll want to pass to other Destiny endpoints.

In this particular case, you need to pass a character ID as well: this is the unique identifier for a specific character on the Destiny Account. To get at that info, you'll have to call an endpoint such as GetProfile (https://bungie-net.github.io/multi/operation_get_Destiny2-GetProfile.html#operation_get_Destiny2-GetProfile). Pass in the "Characters" component, and that'll give you just enough info to get each of the character IDs on the account that you can then use to call this endpoint.

Oy, that was a lot of info, and particularly since we're not exposing GetBungieAccount in the documentation it's a lot more painful and less discoverable of a process than it ought to be. That's a total pain, and I apologize for that. Give those endpoints a try - call them manually at first and see what results you're getting back, once you start getting data back from those it will potentially make more sense. In the meantime, we're here to help - let me know if you run into any problems or have any questions! I'm about to head home for the day so I may be a bit delayed, but I try to check the Github at least once a day, so I'll be here.

[vthornheart-bng]

In the meantime, before I go home I'm going to file a bug to get the documentation fixed to expose GetBungieAccount. It's really essential to call to get linked accounts if you're starting off by querying for Bungie.Net users (which is what the User/SearchUsers/ endpoint does)

[vthornheart-bng]

Alternatively, you can skip GetBungieUser if you let the user choose their platform and give you their platform's Gamertag. You can use SearchDestinyPlayer for that, and that gives you their actual Destiny Membership ID if we find them. But note that it's a very picky "search": it has to be an exact match. (https://bungie-net.github.io/multi/operation_get_Destiny2-SearchDestinyPlayer.html#operation_get_Destiny2-SearchDestinyPlayer)

[joeydood]

I get it - have had to do this kind of api before but... well this is a little nasty! Big ships turn slowly and it seems you have much on your hands here.

I'll give it a shot, maybe tonight or tomorrow. If you're the guy writing the api or managing it, I may be able to help test things for you, i've written apis before, webservices and well tons of other junk in my 40yrs of coding and database stuff.

Thanks much! joeydood

[joeydood]

Thanks dude! Works like a champ, getting the membershipId(userName) - in .net. Now on to the next steps...

public string getMembershipIds(string userName)
{
    try
    {
        string membershipId = "";
        memberInfos mi = getMemberInfo(userName);

        using (var client = new HttpClient())
        {
            client.DefaultRequestHeaders.Add("X-API-Key", apiKey);

            var response = client.GetAsync("https://www.bungie.net/Platform/User/GetBungieAccount/" + mi.userId + "/254/" ).Result;
            var content = response.Content.ReadAsStringAsync().Result;
            dynamic item = Newtonsoft.Json.JsonConvert.DeserializeObject(content);

            //populate return values
            membershipId = item.Response.destinyMemberships[0].membershipId.ToString();

            return membershipId;
        }
    }
    catch (Exception ex)
    {
        return "Error: " + ex.ToString();
    }
}

[vthornheart-bng]

Ah, sweet! I'm glad it is working out, good stuff! Phew - and I'm glad you posted this, otherwise I might have never noticed that we aren't exposing the documentation for one of our most important endpoints. (I'm surprised no one brought it up before! I take the endpoint for granted, but I'm not sure how people found it otherwise!)

[joeydood]

Anytime, my dude. And thank you very much!

The reason I failed to just 'get this' as you noted, is because i was not familiar with the BNet authentication | account | membership | character relationships.

The documentation lacks significant working examples embedded in it eg: microsoft code documentation, including imagery (it is a video game, after all...). I would suggest working examples using a standard language and not make an attempt to support all languages through examples as that will produce more work to maintain over time.

Some overviews of concepts would help also. Your lengthy explanation above was perfect and a great help in educating me.

Again, thank you very much and I'll be around...

[vthornheart-bng]

No problem, glad to help - and indeed, I wouldn't expect anyone to just intuitively get this API! No matter how good a developer is, this particular aspect (the initial "handshake" you have to do to get information) along with the static/dynamic data split are two barriers to entry for anyone just starting to use the API. They're not intuitive, they're not well documented (or in the case of GetBungieAccount, not documented at all! Ugh!), and I wouldn't ever expect anyone to just pick it up in its current state. Also, the D1 API users had an advantage, in that at the time our site was almost entirely client-side rendered; so you could see all of this handshaking process that we did right in our Javascript code. That turned out to be a bad idea for a variety of reasons though, so we moved to a more server-side rendering model... but that has hidden the primary way that people used to be able to discover this information before we had documentation!

We could sorely use more documentation on both points, and examples as well for sure - your suggestions are fantastic, and I agree wholeheartedly.

Indeed, adding to the Documentation has been something we've been kicking down the road in favor of ongoing feature and maintenance work - we've got a lot more than last time (which was basically nothing other than a codegen'd Javascript file that listed all of the endpoints and its inputs/outputs), but we're well short of providing good resources for first time API users in particular. Once a new user gets the core concepts down (particularly the two hairiest ones mentioned above), the documentation is often enough to fish around for what you need... but we need something with more substance for those just learning out.

I'm going to add a separate Github Issue to track that need, because this need has been kicking around in my head for a while but I've not even granted it an Issue yet for tracking. That's got to change if we're ever going to add it!

[vpzed]

There are some community resources that you may want to check out.

Some community docs: http://destinydevs.github.io/BungieNetPlatform/

My personal Wiki on stuff I've put together: https://github.com/vpzed/Destiny2-API-Info/wiki

This is a link to an unofficial Destiny API Discord server: https://discord.gg/HU3Q4up

A blog series on using the API (starts in D1): http://allynh.com/blog/creating-a-python-app-for-the-destiny-api/

[vthornheart-bng]

Good links, thank you @vpzed!

[vpzed]

@vthornheart-bng If you can come up with some topics you might want covered please let the community know. There are a lot more API users than core devs. :)

[vthornheart-bng]

Thank you - your contributions are deeply appreciated @vpzed, and everyone who has been writing documentation out there. Someday we're going to have a meeting of API users, and beer will be on me. ;)

I think #341 covers the bulk of what I'd really love to start conveying to people in a more accessible format. Once someone gets those core concepts, the rest of the API is vast and complicated - but at least more well documented. It's that starting point that's the most rough ride, even more rough than our more complicated API concepts like Talent Nodes and Vendors.

[joeydood]

Thank you both very much for the extra help and information. I will check out the community maintained info to see how to accomplish my intended goal with the api.

Vendal, I will help you as time allows, to create the developer-supported documentation if you like. The fact that i am somewhat ignorant of the BNet api but a well-seasoned developer (haha don't ask...) can be a really good thing.

It's end of year, and i'm between contracts (as they fizzle out by eoy usually) and have wanted to play with the D2 api all year long - so i've got some time on my hands at the moment.

Thanks again for being a resource for this.

[vthornheart-bng]

Thank you, it is deeply appreciated! Yeah, hit up @vpzed, it sounds like he's maintaining that community run documentation - perhaps this can be the beginning of a bigger collaborative effort! I dig it, and it definitely is a relief - admittedly I've had my hands full just working out the kinks in the API and adding features we require, documentation has been getting kicked to the side for quite a while as a result!

[joeydood]

it's always one or the other: good code OR good documentation but never both. tacitly outsourcing it to the community is not a bad thing, especially where there are dedicated and enthusiastic users of the api.

[vthornheart-bng]

Aye - we definitely had to the first time around. I was hoping I was going to get the time this go-round to provide better documentation... and that extra time just hasn't materialized. :(

[joeydood]

That's a 'Productivity by release' issue. P14 of 'Clean Architecture'. Without proper help, it will only grow worse.

[vthornheart-bng]

It's not a productivity by release issue, but rather a manhours-to-backlog-size issue.

[joeydood]

my apologies, i only read up to p15 and thought i knew everything.

[joeydood]

wow.

well i finally got what i was looking for but not sure if it's what i really need just yet.

suffice it to say, i had to write code to figure out the api - which is not very cool at all. while i will not go into details as to refrain from ranting about the api and its documentation because that is in poor taste, i will say this, that your (vendal) hints above were very helpful and made it possible to 'get it' to a certain extent.

Thank you, once again and i hope i have not shot myself in the foot for further help by being critical.

[vpzed]

All of us understand what you are saying because we are all writing the same code to parse the api responses and de-reference all the various values with the world manifest.

One of the challenges that Bungie is trying to meet is making as much stuff in the api a reference so that it can be "internationalized" via the various world manifest localized versions. This alone leads to a lot of the pain points that many people encounter (very opaque api responses, dozens upon dozens of manifest lookups per response, etc).

The other thing to keep in mind is we are essentially piggy-backing on the "backend REST API" for bungie's website and mobile apps. Those are the primary consumers for the design. Yes they provide community focused stuff which we appreciate but you also have to think about what if you wrote your own full-stack website that did what you wanted. Then someone else started using your api for their own purposes - what would be your level of priority for explaining your setup to them versus continuing work on your own site?

As a developer it is easy to begin to think of the destiny api as a part of the game you purchased, but I don't think that is the case. The api is a best-effort, nice-to-have kind of thing. It's super cool what we can do with it as a community, we appreciate the efforts bungie has made with the api and such, but it is best to not think of it as a product "for us".

EDIT: To be clear these are just my opinions.

[vthornheart-bng]

I'm glad you got it working - and I agree that the documentation is lacking, as I mentioned before. @vpzed's recollection of the history of the API is correct: it's grown quite a bit from something we weren't really envisioning supporting officially in any capacity in Destiny 1 to attempting to do so now, and building up both the means of conveying documentation and releasing features that benefit both fan users and our mobile applications and website.

However, that evolution and all of the work we put in (features, bug fixes, and documentation) is going to take time, and no matter what there will be many conflicting priorities in what we can provide in the time we have to work on it.

[joeydood]

yep. work will always take as much time as is required to complete it (old sayin). been doing full stack dev for maybe 20 years or so, global stuff, i get it - that's why i refuse to complain or put forth harsh criticism - what exactly would that accomplish anyway? perhaps make the front page of the new york times: 'mysterious code guru makes rambling tyrannical rant'? or the enquirer: 'hippie ignites code revolt at bungie'.

what 'could' be helpful is a healthy intellectual discussion regarding the issues at hand because well, i've always looked at a problem as a solution in reverse - and the same general mechanisms which create and grow solutions also work for problems because they grow, they take on a life of their own, they fester, etc. so i see a few problems with an api and it just so happens that i mostly can gather the cause of the problem because i also know the solution. not to blow my own horn too much but well, experience teaches those who are willing to learn.

anyhow, i'm just messing around with the api for fun, i've wanted to do something with it for over a year now and here i am. besides, bungie's gonna do as they please. i will say, that it is a rare opportunity for bungie to make available someone like Vendal to respond directly with assistance.

Thanks much.

[floatingatoll]

Please be kinder to the Bungie staff here. I think there's less than a handful of API coders total company-wide and everything they've done has been above and beyond what was necessary to support the official app. That you've been doing full-stack dev for 20 years is nice, but a community issues forum really isn't the place to bring that up. Save that for LinkedIn, and focus instead on contributing useful, practical feedback instead — for example, "There's a gap in your docs about XYZ" or "This structure ABC in your API might be improved by changing it to DEF" or "Your items database has an error" or "The API frequently times out on endpoint GHI".

[joeydood]

roger, ten four, whatevs and stuff.

[joeydood]

Lord Vendor Thunderbrains: I have something but not sure how accurate it is. re: all time stats. would like to submit url privately for your consideration/review.

again, many thanks

[joeydood]

OK well some users are clearly destiny 2 players but there are issues when requesting their stats. Any ideas? Do they HAVE to be bungie members to have stats?

[vthornheart-bng]

Hmm, my apologies, I must have missed the email notification from your last post. If they've played D2 and have completed at least one match, you should be able to get stats from them, but perhaps something else is occurring that we can help with! Can you pass over a membership ID and type of a user for whom you're not receiving stats?

[joeydood]

yes sir, please give me a few to get that for you. SO HAPPY you are replying, is there a way i can give you some info privately as i have only concerns about others' privacy - or is that not really a big deal?

thank you again, Lord Vendor Thunderbrains!

[vthornheart-bng]

Yeah, sorry about that - I would have replied earlier had I seen it, my bad!

Ultimately, the membership IDs, membership Types, and character IDs are all public info: so as long as you don't put anyone's auth token or your API key up here, it's nothing sensitive. No worries.

[joeydood]

i'll get the data you asked for (have to walk the stack for it) but in the mean-time, it would be awesome if you checked out the page i'm writing: joeydood.com

[vthornheart-bng]

Ah, I wasn't able to see stats, but I see what you are trying to do in terms of comparing stats - cool idea! Hopefully we can help with whatever's going awry.

[joeydood]

joeydood.com/stats

[vthornheart-bng]

Ah, yes I went over to that one - plugged in a couple of gamertags (and then a couple of membership IDs just to see if that would work) but didn't get results. but I think the idea is cool! I may also be plugging in the wrong thing: the gamertags I plugged in were two PS4 users whose gamertags I knew.

[joeydood]

one member in particular is failing: cm_punk290366 i am working to get you the details you require. hadn't realized just how much code i wrote!

[vthornheart-bng]

Ah, awesome - I will take a peek when I get some time, I'm in the midst of some other tasks but I'll do a little digging as soon as I can! In the meantime, yeah send the URL you're passing and what you're getting back when it fails so I see specifically what you're calling.

[joeydood]

understandable, will do. it will take a little time. gotta write the code that pays the bills first!

again, thank you very much! Long Live Lord Vendor Thunderbrains!

[joeydood]

i think i may have figured out the issue. it appears that there are some null values either being returned by the api or evaluated by my code and i need to handle that. damn nulls. it could be the json deserialization too. it's a 'my problem' it seems though.

i'm getting this from "cm_punk290366", response.raid.alltime.kills.basic.value.

[vthornheart-bng]

Ah, no worries - aye. Let me know if you run into anything else, we'll be here! (it may take a bit to respond if I'm knee deep in something else, but I'll try to respond when I can!)

[joeydood]

thanks!

my guess is the user has not yet engaged in the activities which are evaluated as null. i've asked him but haven't heard back yet.

[joeydood]

Ok, nailed it (i think) - one issue i was encountering was the null vals but some issues still need to be addressed such as the difference between a user's ps4 name and their destiny 2/bungie name.

also, addressing 'ties' because rare as it may be, it can happen.

[joeydood]

This is working out nicely so far.

New additions today:

• Added Bootstrap - made site work for mobile.
• Added Custom Exception Messages for null response object and null stats objects.
• Added Tab Controls for better visibility.
• Added by custom request: Remaining Time After Quit Seconds (aka Rage Quits) [don't quote me on that eh? i'm not sure what that stat really means].
• Fixed Ties - when two players have the same value for a stat.

All Hail Lord Vandor Thunderbrains!