Getting basic item information

[afehners]

Using Destiny2.GetItem I can get all sorts of information about an instance of an item. However I don't seem to be able to get simple things such as the name or type of the item.

In D1 I would take the item hash and make a call to Destiny/Manifest/{itemType}/{ItemHash}/ to get this basic item information.

Is the only way to get basic item details in D2 to download the SQLite manifest zip and retrieve these details directly from it? If so it would be really good to have some of this basic information returned through the API. It adds complexity to keep the local copy of the file in sync for only a few extra attributes of an item and adds a barrier to entry for new App developers looking to use your API.

[sgtfrankieboy]

This is a duplicate of Issue #10.

Edit: To answer your question, yes the Sqlite DB currently the only legit way of getting the D2 Manifest data.

[afehners]

So does that mean the only way to get an item's name, and other static info, is through downloading the Manifest and querying the SQLite database locally?

Edit: Thanks for the answer, that's a shame. Some really good changes to the API, but this bits the ugly duckling!

[vthornheart-bng]

Oy, indeed - this is unfortunately the biggest barrier to entry for using the API at the moment.

Issue #10 is one improvement I'd like to make, for people who truly have only superficial-level desire for access of entities (i.e. just their name and such, and not any related entity data). I am not certain yet how many use cases people will have for that, but I would like to provide that shortly.

I also would like to provide the manifest data in a more convenient format, like straight-up statically hosted JSON files. That's been on my mind for a while, because the Mobile Manifest database really only exists in that format because, before Destiny 1 launched, our mobile engineers found it to be a convenient format for their purposes. Now that more people are using the API, having formats more convenient for others would be great!

What we unfortunately can't go back to is returning definition information with "live data" responses as we allowed people to do in Destiny 1. That would be the most convenient from a user standpoint: but that data can balloon into tens of megabytes per response (or more!), which burdened both ourselves and our users. To work with the details of Destiny data, you need not just the entity itself but all of its related entities - it quickly gets out of hand in terms of the size of data you need. Once you do the one-time setup of creating a system to manage and update your static manifest data, you'll find the benefits of convenient local access are significant... but it is an unfortunate first step, and one that we need to provide better documentation for.

But, I would like to provide the improvements mentioned above. It is still unfortunately a barrier to entry, but one that we'll have to accept for the time being.

[floatingatoll]

If scoped only to the field "object localized name" e.g. "Gun Name Here" or "Armor Name Here", can that single field be optionally included with "live data" responses without ballooning response sizes appreciable?

There's definitely a slippery-slope problem, but I'd like to explicitly confirm that just the object name is, alone, enough to overwhelm the API with megabytes of extra data?

On Thu, Sep 7, 2017 at 9:37 AM, Vendal Thornheart notifications@github.com wrote:

Oy, indeed - this is unfortunately the biggest barrier to entry for using the API at the moment.

Issue #10 https://github.com/Bungie-net/api/issues/10 is one improvement I'd like to make, for people who truly have only superficial-level desire for access of entities (i.e. just their name and such, and not any related entity data). I am not certain yet how many use cases people will have for that, but I would like to provide that shortly.

I also would like to provide the manifest data in a more convenient format, like straight-up statically hosted JSON files. That's been on my mind for a while, because the Mobile Manifest database really only exists in that format because, before Destiny 1 launched, our mobile engineers found it to be a convenient format for their purposes. Now that more people are using the API, having formats more convenient for others would be great!

What we unfortunately can't go back to is returning definition information with "live data" responses as we allowed people to do in Destiny 1. That would be the most convenient from a user standpoint: but that data can balloon into tens of megabytes per response (or more!), which burdened both ourselves and our users. To work with the details of Destiny data, you need not just the entity itself but all of its related entities - it quickly gets out of hand in terms of the size of data you need. Once you do the one-time setup of creating a system to manage and update your static manifest data, you'll find the benefits of convenient local access are significant... but it is an unfortunate first step, and one that we need to provide better documentation for.

But, I would like to provide the improvements mentioned above. It is still unfortunately a barrier to entry, but one that we'll have to accept for the time being.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Bungie-net/api/issues/62#issuecomment-327855063, or mute the thread https://github.com/notifications/unsubscribe-auth/AAFqDGDqrv3UNNrnvyaODvtd3e5n-9Hfks5sgBu_gaJpZM4PPmrT .

[vthornheart-bng]

There's other reasons why we don't put one-offs like that in the data unless they're hashes, numbers, or pointers to images: if you look through the live datasets, you'll see that they've very carefully and very intentionally been crafted to be language agnostic. It provides us a number of performance, caching, and infrastructure simplification improvements, and also allows you all to query the API in a language agnostic manner (while grabbing any language-specific information you need from your chosen manifest database(s)). We had a workaround for that to return definitions in Destiny 1's API when we allowed the definitions=true querystring param to be passed, but it was pretty ugly, removed our ability to output cache once for all languages, and I'd rather not do it just for one-off localized properties.

[sgtfrankieboy]

When doing that it will happen quickly that people are going to request more stuff, you might add name, icon and tier but then someones wants description, etc.

I think the best solution would be to bring a /platform/destiny2/manifest/{type}/{hash} endpoint for people who do not want/can't setup backend to load the Sqlite manifest. Maybe even have a POST method for batch requesting data (however that would be hard to cache, but its more about the idea)

[vthornheart-bng]

Yeah, I'm definitely down for adding just such an endpoint as you mention here.

I'm also not immediately opposed to batch requests - I just would hate to see it turn into a similar debacle to ?definitions=true, where people set up anti-patterns that cost them (and us, heh) lots of bandwidth for the sake of convenience.

[vthornheart-bng]

Ultimately, my hope is that, if we can make it easier to get at the manifest data - i.e. via statically hosted JSON files instead of the zipped up SQLite nonsense - that might reduce the barrier to entry enough that it could be more convenient... and it'll never be as convenient as being able to hit the web service every time for that data, but I'd like it to at least be convenient enough that people would think twice about hitting the web service.

[floatingatoll]

?titles=true would be fine for casual users, and provides a very clear boundary of "we're not expanding on this".

On Thu, Sep 7, 2017 at 10:17 AM, Vendal Thornheart <notifications@github.com

wrote:

Yeah, I'm definitely down for adding just such an endpoint as you mention here.

I'm also not immediately opposed to batch requests - I just would hate to see it turn into a similar debacle to ?definitions=true, where people set up anti-patterns that cost them (and us, heh) lots of bandwidth for the sake of convenience.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Bungie-net/api/issues/62#issuecomment-327865524, or mute the thread https://github.com/notifications/unsubscribe-auth/AAFqDCE4i_TXj5oF1816y-6vDcaQItilks5sgCTWgaJpZM4PPmrT .

[ckhicks]

+1 for all of this so far. Having the definitions returned inline was a mess, but being able to quickly call item details by name/hash/etc. would be super helpful for on-demand results vs updating local copies of the manifest via bulk methods on a timed basis. Thanks so much for being willing to flex the system for easier use!

[vthornheart-bng]

It's tempting, floatingatoll... but the dance we'd have to do on the backend to accommodate it wouldn't be worth the effort - and indeed I have a hard time picturing someone wanting (for example) just the names but not the descriptions, or once they get those wanting the icons (which at least the icons are a bit easier to accommodate with our existing infrastructure, since they're language agnostic, but I digress). Sorry, but I'm not going to add any localized properties to Destiny live data requests - but I'm glad to do what I can to make it more accessible, including #10 !

[afehners]

Static json files per table would be a good step forward above pulling in an SQLite db where it isn't immediately obvious how the index field relates to the ItemHash (unsigned int vs signed)

I think in most cases when someone calls Destiny2.GetItem there are some static fields that they would want to get out of the manifest for it to be useful, such as how do you display info about an item without the name to at least display. On Destiny2.GetItem I initially thought this more general static information would be returned when component ItemCommonData was requested,but it does not.

All the item data I need is at the root of the Manifest for the item, would it be possible to return this top level data when ItemCommonData is added to the component list for Destiny2.GetItem?

The fields I'm referring to are:

{"displayProperties":{
"description":"This weapon was forged by Devrim Kay VIII for the survivors.", "name":"Hand in Hand", "icon":"/common/destiny2_content/icons/fd43acca1548271b8e0ea8e8c4d75783.jpg", "hasIcon":true }, screenshot":"/common/destiny2_content/screenshots/4138415948.jpg", "itemTypeDisplayName":"Shotgun", "itemTypeAndTierDisplayName":"Uncommon Shotgun", "displaySource":"", "allowActions":true, "nonTransferrable":false, "specialItemType":0, "itemType":3, "itemSubType":7, "classType":3, "equippable":true, "defaultDamageType":3, "defaultDamageTypeHash":1847026933, "hash":4138415948, "index":1671, "redacted":false}

I think those fields cover the basic requirements of fetching an Item for most, and any further information could be pulled from the statically hosted json files and cached locally.

Edit: Read your point about localise text in with live data and can see how that wouldn't be great. Perhaps the data above could be returned via /platform/destiny2/manifest/{type}/{hash} to keep it light weight and any further data is fetched via static json files.

[vthornheart-bng]

Yeah, for sure the /platform/destiny2/manifest/{type}/{hash} endpoint will return those: I've got no beef with that returning the whole DestinyItemDefinition as long as it's a separate endpoint. (and that way I don't make too many assumptions about what the end user is actually going to be using: that was basically how all of our overly-specialized endpoints in D1 ended up coming into existence, when we were assuming how people were going to use our data, or assuming that their use cases were similar to ours)

A single DestinyItemDefinition won't break the bank, even a large one is no more than a few KB on its own - so returning the whole thing from that new endpoint won't feel bad to me. I just wouldn't want to return related entities, and I wouldn't want to set another precedent of returning localized data through the live data endpoints. So I think this'll work out once that new endpoint exists! It's not the most convenient scenario, but hopefully it will be useful enough to work with.

[vthornheart-bng]

I'll close this one now in favor of issue #10, but definitely feel free to continue the conversation over there!