Collect Lifetime Changes to Item Stats from Infusion

[SunburnedGoose]

I need a feature of DIM that can collect the stats for items and track their changes as the item is infused. These changes will help understand the rate of change each item experiences as items are infused into it. The question I'm looking to answer is do all items of similar type have the same rate of change?

We'll need a couple of features to make this happen.

1. This will be an opt-in feature which can be toggled in settings.
   1. The user will be prompted to enable this feature the first time they use DIM after its release.
2. A signal to turn on the feature.
   1. The feature is disabled by default.
   2. The feature will be controllable at the start of DIM from a remote resource.
3. The capability to locally storage as the resource to save the data.
4. The feature will save the base stat values for all items with defense ratings.
5. DIM will update local storage and insert the latest stats for an item when the defense value of the instanceId does not match the value in local storage. This is considered to be an infusion event.
6. An item is considered to be complete when:
   1. It was infused at least once.
   2. When one of the following are true:
      1. The instanceId of an item in local storage is not found in the latest update from the Destiny API. This means the item was deleted from a character or infused into another item.
      2. The item has been infused and the defense value is > 332.
      3. The sum of the stats is at the max CSP for that item type.
7. DIM will upload the completed items and their stat history to a remote resource for analysis.
   1. The data will be anonymous. No account or character data will be sent.
   2. Only a small sample of data will be uploaded to the remote resource. We do not need every infusion event from all users uploaded. Let's start with 1% chance to upload all completed items when a completed item event occurs.

The remote resource should be a restful endpoint that accepts data to be stored into a database.

Questions

1. Do we have a means to prevent someone tampering with the data that is sent to the endpoint?
2. Should we make the percentage of data events captured configurable remotely?
3. Firebase? Azure? Where should we save this data?

[rowanbrendan]

• How do 6i and 6iii differ? 6iii seems like a more specific version of 6i.

• Regarding 6iv,

  The sum of the stats is at the max CSP for that item type.

  The item should be complete if the sum of that stats is at or greater than the max CSP, in case we're caught off guard.

• And then in 7ii,

  Let's start with 1% chance to upload all completed items when a completed item event occurs.

  Be careful about uploading duplicate data, though. Need to locally keep track of infusion events uploaded.

At this point I don't have answers to your three questions, but everything else looks good. Question 1 is important.

[SunburnedGoose]

We'll remove the data once it's uploaded. There won't be duplicates.

I've updated 6 to hopefully clear it up. Just trying to make it clear that an infusion event must occur, so when an item is deleted, we can upload that data point.

As for tampering, that's a tough one.

[kyleshay]

fixed in #572 per work from the data collecting script. can re-run the script if we need to update values, don't need it built into DIM at this time?