HyperdictionStat

[NoFoolLikeOne]

Get hyperdiction locations and dates from commander logs via EDMC. EDMC will only supply data if the the stats have changed.

Requires new Strapi Report model

Requires aggregation into strapi HyperdictionSite. model

Question: Should we keep the old model with from and to sites and the hyperdiction detector which is a bit unreliable?

Or should we move to the more reliable stats model but risk missing hyperdiction data when hyperdictions happen in quick succession?

[NoFoolLikeOne]

@derrickmehaffy what do you think? I'm inclined to think that the Hyperdiction detector is too unreliable and creates to much work when things go wrong.

But potentially we could track both and use the stats to validate a detected hyperdiction. Eg if we detected a hyperdiction and the last hyperdiction stat is older then that proves it was a false positive. If the dates match then we can confirm that it was a genuine hyperdiction. But there will be situations where we cannot confirm.

[derrickmehaffy]

Hmmmmm. What do the stats look like from EDMC? I haven't seen these

[NoFoolLikeOne]

The statistics event has all of this, but we are only interested in the Thargoid Encounters Data (in bold) and only if it has changed

{ "timestamp": "2018-10-07T19:30:47Z", "event": "Statistics", "Bank_Account": { "Current_Wealth": 1228358633, "Spent_On_Ships": 530017802, "Spent_On_Outfitting": 1234472123, "Spent_On_Repairs": 17873848, "Spent_On_Fuel": 545333, "Spent_On_Ammo_Consumables": 2850309, "Insurance_Claims": 175, "Spent_On_Insurance": 414362579 }, "Combat": { "Bounties_Claimed": 469, "Bounty_Hunting_Profit": 12105736.75, "Combat_Bonds": 4015, "Combat_Bond_Profits": 42236400, "Assassinations": 0, "Assassination_Profits": 0, "Highest_Single_Reward": 299090, "Skimmers_Killed": 326 }, "Crime": { "Notoriety": 0, "Fines": 273, "Total_Fines": 3761356, "Bounties_Received": 571, "Total_Bounties": 139655, "Highest_Bounty": 6825 }, "Smuggling": { "Black_Markets_Traded_With": 23, "Black_Markets_Profits": 5963748, "Resources_Smuggled": 1141, "Average_Profit": 87702.176470588, "Highest_Single_Transaction": 602252 }, "Trading": { "Markets_Traded_With": 144, "Market_Profits": 32466857, "Resources_Traded": 33223, "Average_Profit": 47258.88937409, "Highest_Single_Transaction": 7058310 }, "Mining": { "Mining_Profits": 1836908, "Quantity_Mined": 574, "Materials_Collected": 9443 }, "Exploration": { "Systems_Visited": 9504, "Exploration_Profits": 261926116, "Planets_Scanned_To_Level_2": 627, "Planets_Scanned_To_Level_3": 39996, "Highest_Payout": 2987546, "Total_Hyperspace_Distance": 336964, "Total_Hyperspace_Jumps": 12532, "Greatest_Distance_From_Start": 21949.760833536, "Time_Played": 6362880 }, "Passengers": { "Passengers_Missions_Accepted": 280, "Passengers_Missions_Disgruntled": 16, "Passengers_Missions_Bulk": 1542, "Passengers_Missions_VIP": 632, "Passengers_Missions_Delivered": 2174, "Passengers_Missions_Ejected": 20 }, "Search_And_Rescue": { "SearchRescue_Traded": 316, "SearchRescue_Profit": 9268968, "SearchRescue_Count": 59 }, "TG_ENCOUNTERS": { "TG_ENCOUNTER_WAKES": 7, "TG_ENCOUNTER_IMPRINT": 17, "TG_ENCOUNTER_TOTAL": 17, "TG_ENCOUNTER_TOTAL_LAST_SYSTEM": "Pleiades Sector OI-S b4-0", "TG_ENCOUNTER_TOTAL_LAST_TIMESTAMP": "3304-09-15 07:35", "TG_ENCOUNTER_TOTAL_LAST_SHIP": "Krait MkII", "TG_SCOUT_COUNT": 74 }, "Crafting": { "Count_Of_Used_Engineers": 13, "Recipes_Generated": 714, "Recipes_Generated_Rank_1": 140, "Recipes_Generated_Rank_2": 137, "Recipes_Generated_Rank_3": 188, "Recipes_Generated_Rank_4": 151, "Recipes_Generated_Rank_5": 98 }, "Crew": { "NpcCrew_TotalWages": 14953969, "NpcCrew_Hired": 7, "NpcCrew_Fired": 2, "NpcCrew_Died": 4 }, "Multicrew": { "Multicrew_Time_Total": 14603, "Multicrew_Gunner_Time_Total": 1429, "Multicrew_Fighter_Time_Total": 0, "Multicrew_Credits_Total": 2370278, "Multicrew_Fines_Total": 0 }, "Material_Trader_Stats": { "Trades_Completed": 35, "Materials_Traded": 478, "Encoded_Materials_Traded": 50, "Raw_Materials_Traded": 88, "Grade_1_Materials_Traded": 9, "Grade_2_Materials_Traded": 94, "Grade_3_Materials_Traded": 88, "Grade_4_Materials_Traded": 114, "Grade_5_Materials_Traded": 173 }, "CQC": { "CQC_Credits_Earned": 922, "CQC_Time_Played": 900, "CQC_KD": 0.28571428571429, "CQC_Kills": 2, "CQC_WL": 0 } }

[NoFoolLikeOne]

I use it to produce the stats on this spreadheet

https://docs.google.com/spreadsheets/d/16u--KbRoQzQ22fFqVxrJlNpQWCPiPK8SDAP6osaWYok/edit?usp=sharing

And charts like this one

[derrickmehaffy]

Will need to think on this and if we want to keep recording this data in the API due to complexity and false positives.

[NoFoolLikeOne]

If we get the hyperdictions from statistics then it is guaranteed no false positives. But we wont have "to" and "from" systems and we wont capture all hyperdictions.

If we wanted to keep the "to" and "from" then we could make the detector manual entry only. Eg prompt

Were you hyperdicted in HR 1187? (Yes|No)

[NoFoolLikeOne]

Been looking at the hyperdiction data and it looks like the current detector is stable and not making any false positives. There are nearly 900 systems with recorded hyperdictions since we started.

The data captures from the stats screen also seems to be doing quite a good job of recording hyperdictions too and has already identified 296 sites. Bear in mind that it captures hyperdictions from before the user installed the software.

The key question is whether we want to capture any manual data eg how many thargoids and what types were present etc. I don't know if anyone actually looks at that manually entered data now that we have had over a year of hyperdictions

I'd say that if we are no longer interested in capturing manual info then we can scrap the detector and just use the thargoid encounter stats

If we still want to capture data manually then we need to decide what we want to capture

[derrickmehaffy]

I think the less user input the better to streamline things, maybe we should get the thoughts of the Thargoid Boffins?

[NoFoolLikeOne]

The Hyperdiction sheet here is what were were accepting manually from PC and console players

https://docs.google.com/spreadsheets/d/1IA3HxotFUXh9qJi3v-wtcenvMF-znamfQ8JtNJbiZdo/edit?usp=sharing

It doesn't seem all that useful but nobody has done any analysis on it as far as I know

[NoFoolLikeOne]

I have made a decision.

We will use the hyperdiction stats as the primary source of information and will not record the jump destination in the new interface.

We will also accept hyperdictions that are detected by the plugin but only if confirmed by the users. We will record how that data was supplied. eg: Stats/Plugin/Manual Console players should also be allowed to submit hyperdictions through a form

The aim will be to build up a map of where hyperdictions occur. If possible alert when something new happens, eg first time a hyperdiction has occurred in a system or if the hyperdiction is closer to earth than any other.

We will not bother with any other manually entered data on hyperdictions at this time.

Some reasoning that may form the basis of an article for canonn.science here https://docs.google.com/document/d/19E9D0OvIRNVxgwCT88JiLANujjU7A_MosomnqcrzjhY/edit

Please request edit if you want to contribute to it.

If we decide we want to record more information then that can be done under a separate model. For instance thargoid glyphs could be associated with a hyperdiction event a surface site or a USS. As can thargoid encounters.

[NoFoolLikeOne]

When we record a hyperdiction from either the tgencounterstats or from the hyperdictionreport we trigger a script.

update systems model with system and edsm data

if systemid is in hyperdictions model then update the count else insert into the hyperdictions model with count=1

find the closest system to sol from the hyperdictions model. If the new site is closer then send a message to discord. (CMDR Blah has been hyperdicted nnly from Sol, this is a new record for Hyperdictions)

find the furthest system from Merope from the hyperdictions model. If the new site is further then send a message to discord. (CMDR Blah has been hyperdicted nnly from Merope, this is a new record for Hyperdictions) only do this if not closest to Sol.

NB: How do we stop duplicate reports if we accept manual submissions?

We also want to update commander stats but again, do we use TGencounter stats or increment? Maybe the answer is to completely retire the hyperdiction detector?

.

[derrickmehaffy]

going for this completely retire the hyperdiction detector as we really don't care about HD's anymore