[Request] bulk PGCR or add more necessary data to getActivityHistory

[SarKurd]

I know it's been already asked to return bulk PGCR but still want to give it another try

i'm trying to get players badges like how Raid.report does it but having a hard time to process all those PGCRs as it takes way too long and i have to throttle the requests too, there are players that have thousands of clears(example: SPARK XBL based on raid.report stats).

I was hoping you could add bulk PGCR or add necessary raid values to getActivityHistory or come up with another solution

things i need for my app at this moment in getActivityHistory

1- Number of players combined that completed the activity
2- StartingPhaseIndex
3- Number of deaths combined for players that completed the activity

That would save us from renting a DB and decreases number of requests to the API.

I know that's a big request

[vthornheart-bng]

Aye, unfortunately I don't see us being able to provide this any time soon with the resources devoted to the API and the time we have between this and other projects.

From what it sounds like you're doing - is it that, when you look at a given persons' info, you go and fetch their specific activity history and the PGCRs for them on demand? If that's what you're doing, the idea of storing it in a database (though it does incur a cost) or even flat files (though it may incur a cost for you) may at least save you from re-processing costs for frequently accessed accounts. There's some other alternatives you could take if need be as well, such as "scraping" every PGCR instead of per character - though all of the alternatives I can think of involve something being stored in some way on your end to save yourself from duplicated & expensive requests.

A long term solution I'd like to see is hosting PGCR data as static files that you could then access cheaply from our CDN, or that perhaps we'd even bundle - imagine historical data in daily/monthly ZIP files or some such. But unfortunately right now with the time we have available, that's a distant daydream of mine. :( We'll keep you all posted here if the situation changes - and I wish I had better news to provide on that front.

[floatingatoll]

As a temporary measure, if you were to simply post a list of PGCR IDs each day, and let people go fetch their own JSON from the static endpoint, that would certainly be light-years more efficient without (as much) burden. Probably not less enough to be possible, but worth a shot suggesting anyways.

On Jun 24, 2019, at 15:40, Vendal Thornheart notifications@github.com wrote:

Aye, unfortunately I don't see us being able to provide this any time soon with the resources devoted to the API and the time we have between this and other projects.

From what it sounds like you're doing - is it that, when you look at a given persons' info, you go and fetch their specific activity history and the PGCRs for them on demand? If that's what you're doing, the idea of storing it in a database (though it does incur a cost) or even flat files (though it may incur a cost for you) may at least save you from re-processing costs for frequently accessed accounts. There's some other alternatives you could take if need be as well, such as "scraping" every PGCR instead of per character - though all of the alternatives I can think of involve something being stored in some way on your end to save yourself from duplicated & expensive requests.

A long term solution I'd like to see is hosting PGCR data as static files that you could then access cheaply from our CDN, or that perhaps we'd even bundle - imagine historical data in daily/monthly ZIP files or some such. But unfortunately right now with the time we have available, that's a distant daydream of mine. :( We'll keep you all posted here if the situation changes - and I wish I had better news to provide on that front.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[vthornheart-bng]

Unfortunately, with the way the stats system is architected we can't easily push those new IDs externally without changes to a system owned on the game side that is much slower to iterate on than Bungie.net itself. We don't have knowledge of new PGCR IDs that were created right now, and there'd be a (higher) cost to exposing it in that underlying system.

And the (smaller, but still more than we have time to do right now) cost of building something that would periodically create and publish those lists.

[SarKurd]

Unfortunately i can't afford that, at this moment i really on the periods for each activity, and save the last activity period for each player in my DB so i don't have to fetch the old ones as i have already checked them.

I try to make my DB size as small as possible so i can still use the free sandbox plan :-D

I'm glad that you are already presenting a solution, hopefully we will see it in the future.