virtualcommons / port-of-mars-analysis

R analysis pipeline for Port of Mars "Mars Madness" tournaments
https://portofmars.asu.edu
GNU General Public License v3.0
0 stars 0 forks source link

issues with events data #9

Closed MarcoAJanssen closed 7 months ago

MarcoAJanssen commented 2 years ago

The data file that includes information on events that happened in the round has a number of issues:

MarcoAJanssen commented 2 years ago

I looked more into the sandstorms and say that for games where there could be 3 rounds of sandstorms (game does not end before 3 rounds), there are 3 sandstorm events for games 80, 83, 87, 88, 93 and 100, but only 2 sandstorm events for games 64, 67, 69, 74 , 90,95 107 and 109.

game 79 has no events recorded for round 2 game 91 has no events recorded after round 8 game 95 has no events recorded for rounds 2 and 3 game 105 has no events recorded for round 2 and 3 game 107 has no events recorded for round 2 and 3

So the issues happened in both Fall and Spring data sets

alee commented 2 years ago

Looks like there is a bug in the way game events are being summarized, looking at game 64 there is clearly an events phase in round 8 and round 9. For round 8, accomplishments were forced to be discarded so the event that forces people to discard one purchased accomplishment. And in round 9 it looks like perhaps it was personal gain, chat indicates that some players wanted the group to "vote no".

I'm not sure what the deal is with the summarizers at the moment and I probably won't have time until Friday afternoon to look into the data issues. For the moment treat the processed data as unreliable - the "true" data can be found in the processed/gameEvent.csv file but that is probably unhelpful for analysis, but more of a sanity check.

alee commented 2 years ago

Should be fixed as of https://github.com/virtualcommons/port-of-mars/releases/tag/v2022.04 and updated data has been uploaded to the Google Drive

MarcoAJanssen commented 2 years ago

I checked the new data files, and here are some observations:

alee commented 2 years ago
  1. ignore game id 110, this data export was run on a local machine that had loaded in the data from production and had test data as well.. I'll rerun the analyses to have this removed when we publish the final data in OSF
  2. that doesn't appear to be included in the analysis scripts that Calvin created and currently would need to be manually collected from the raw data file gameEvent.csv. For example, in game 87 round 7:
87 | vote-hero-or-pariah | {"role":"Politician","heroOrPariah":"pariah"}
87 | vote-hero-or-pariah | {"role":"Politician","heroOrPariah":"hero"}
87 | vote-hero-or-pariah | {"role":"Researcher","heroOrPariah":"hero"}
87 | sent-chat-message | {"role":"Pioneer","round":7,"message":"no   pariah we need the helth","dateCreated":1645915329611}
87 | vote-hero-or-pariah | {"role":"Pioneer","heroOrPariah":"hero"}
87 | sent-chat-message | {"role":"Researcher","round":7,"message":"right","dateCreated":1645915339503}
87 | vote-hero-or-pariah | {"role":"Curator","heroOrPariah":"hero"}
87 | set-player-readiness | {"role":"Curator","value":true}
87 | vote-hero-or-pariah | {"role":"Entrepreneur","heroOrPariah":"hero"}
87 | vote-hero-or-pariah-role | {"role":"Pioneer","vote":"Pioneer"}
87 | vote-hero-or-pariah-role | {"role":"Entrepreneur","vote":"Entrepreneur"}
87 | vote-hero-or-pariah-role | {"role":"Politician","vote":"Entrepreneur"}
87 | vote-hero-or-pariah-role | {"role":"Researcher","vote":"Researcher"}

I need more details for the third question, is this just for the researcher in a specific game or across all data entries? The previous data was being generated / encoded incorrectly after the switch in November 2021 when a lot of questions were moved from the after-round-1 pregame survey into the round 1 pregame survey. There was also one duplicate entry where a participant submitted two survey entries and we had to pick one so it's possible we picked the entry with less data.

We can look into adding an additional summarizer that collects interactive event information like hero or pariah or personal gain - is personal gain being collected properly?

MarcoAJanssen commented 2 years ago

Yes it would be helpful to know the outcomes of votes during events (pariah or hero, personal gain votes)). We did not change the survey after Fall 2021 Mars Madness, but it is good to hear that the problem seems to be resolved. I only have seen missing information for this one pregame survey by the researcher in game-id 67