LAMP Cognitive Games Changes

[carlan1]

There are two changes we would like made to the recently made cognitive games:

Iowa task / wheel spinning game:

• When users manually exit the game, the data/results are not saved in the server. We would like all data saved, provided they completed the task (in the current implementation, if you complete the task, but manually exit, the data is not saved)
• It appears that when users click the buttons very quickly, some of the answers aren't saved. Notice how some of the items (4-9) are not present:

  [{'duration': '0.803s',
  'item': 1,
  'level': 2,
  'type': 2000,
  'value': 0},
  {'duration': '0.817s', 'item': 2, 'level': 2, 'type': 2050, 'value': 0},
  {'duration': '0.836s', 'item': 3, 'level': 2, 'type': 2050, 'value': 0},
  {'duration': '0.845s', 'item': 10, 'level': 2, 'type': 2000, 'value': 0},
  {'duration': '1.238s', 'item': 11, 'level': 2, 'type': 1950, 'value': 0},
  {'duration': '0.8s', 'item': 13, 'level': 2, 'type': 1950, 'value': 0},
  {'duration': '0.814s', 'item': 14, 'level': 2, 'type': 1900, 'value': 0},
  {'duration': '0.8s', 'item': 15, 'level': 2, 'type': 1900, 'value': 0},
  {'duration': '0.867s', 'item': 16, 'level': 2, 'type': 1900, 'value': 0},
  {'duration': '0.819s', 'item': 17, 'level': 2, 'type': 1850, 'value': 0},
  {'duration': '1.462s', 'item': 18, 'level': 2, 'type': 1900, 'value': 0},
  {'duration': '0.878s', 'item': 19, 'level': 2, 'type': 1900, 'value': 0},
  {'duration': '1.541s', 'item': 20, 'level': 2, 'type': 1900, 'value': 0}]

Balloon risk:

• Like the wheel spinning game, we would like entries to be saved even when exit is done manually by the user (if they complete the task)
• It appears that the balloon popping limits aren't random / don't follow the distribution as indicated by the settings. Please check this

[sarithapillai8]

@carlan1 @ZCOEngineer @michaelmenon @avaidyam We will be saving the data for all the games(if there is actually any activity event data) on manually clicking the back button. No confirmation dialog will be shown. Surveys/DBT/Journal will not have this option. For those activities, the activity events will be saved by clicking save./submit buttons.

[carlan1]

Yes. For mindLAMP games, clicking the back button manually should save the ActivityEvent

[carlan1]

Here is an overview of what is desired for this issue as well as https://github.com/BIDMCDigitalPsychiatry/LAMP-platform/issues/763

Games:

• No need to store data locally.
• If a participant exits the app, or manually exits the game within the app, save the data as an ActivityEvent
• When they re-enter, restart the game
• Pressing the exit button will not require a dialogue

Text-based surveys, journals, etc:

• Save the data locally
• If they exit the app, and re-enter, give a dialogue asking to choose between 1. restarting or 2. resuming with the locally saved data
• If they manually exit the activity without pressing the submit button, do not save the activityevent, but save the data locally and prompt the same dialogue as above if they re-enter
• Do not create an ActivityEvent until the activity has been submitted/completed

These should be implemented in addition to the above issues regarding balloon risk randomness and some of the spins from spin the wheel not being saved

[divyav2020]

We have checked the issue, "It appears that the balloon popping limits aren't random / don't follow the distribution as indicated by the settings. Please check this"

Zco: We have checked the issue and the balloon popping is according to the settings. These are our observation.

1. Balloon Game 1: Breakpoint Mean=60.5, Breakpoint Standard Deviation=30 (Number of Balloons: 10) 79 81 73 71 72 78 79 75 76 78

2. Balloon Game 2: Breakpoint Mean=64.5, Breakpoint Standard Deviation=37 (Number of Balloons: 10) 83 87 79 79 83 79 77 82 85 79

3. Balloon Game 3: Breakpoint Mean=70.5, Breakpoint Standard Deviation=37 (Number of Balloons: 10 ) 90 90 89 91 91 82 90 89 88 81

4. Balloon Game 4: Breakpoint Mean=70.5 ,Breakpoint Standard Deviation=40 (Number of Balloons: 10) 93 93 94 82 85 96 87 95 95 90

Please Confirm.

[divyav2020]

Iowa task / wheel spinning game: It appears that when users click the buttons very quickly, some of the answers aren't saved. Notice how some of the items (4-9) are not present:

This issue is fixed and updated in dashboard staging. Please Confirm

[michaelmenon]

Hi @divyav2020 , @sarithapillai8 I think we should check the equations we are using , the values seems incorrect. we need to update

[carlan1]

The fix for the Iowa task is functional in staging

[divyav2020]

We have updated following changes in Dashboard staging:

1. When users manually exit the game, the data/results are not saved in the server. We would like all data saved, provided they completed the task (in the current implementation, if you complete the task, but manually exit, the data is not saved)

2. It appears that when users click the buttons very quickly, some of the answers aren't saved. Notice how some of the items (4-9) are not present:

3. Like the wheel spinning game, we would like entries to be saved even when exit is done manually by the user (if they complete the task)

Pending:

1. It appears that the balloon popping limits aren't random / don't follow the distribution as indicated by the settings. Please check this - We are working on this. We will update once it is fixed.

[carlan1]

To clarify, if a participant exits a cognitive game / task (by pressing the exit button on mindLAMP), and re-enters the game, the game should restart from the beginning (i.e. progress should not be saved).

[ZCOEngineer]

@carlan1 noted.

[carlan1]

@ZCOEngineer @sarithapillai8 We talked yesterday about whether we want to maintain the existence of partial ActivityEvents or if, alternatively, we define a completion condition for each game and only save the data as an ActivityEvent IF the completion condition has been met.

We have decided NOT to impose the completion condition for saving ActivityEvents (i.e. it is still OK to save partial ActivityEvents). However, to improve downstream data analysis, please add some marker to indicate whether a cognitive game had a manual exit from the user. Something like {'type': 'manual_exit', 'value': True} or {'type': 'manual_exit', 'value': False} as a temporal_slice would be acceptable, unless you propose a better alternative.

If a participant exits a game manually (by pressing the back/exit button in the mindLAMP UI) OR the participant closes the app, please mark the value:true.

cc @avaidyam

[sarithapillai8]

@carlan1 We can add {'type': 'manual_exit', 'value': True} on manual exit.

the participant closes the app entirely (i.e. local memory has been cleared and the game cannot resume), please mark the value:true. -

We are not saving game data locally, Local save was decided only for survey, dbt and journal. So if the participant closes the app, no data will be saved. Do you want to change this?

[carlan1]

You are correct. I have clarified my prior comment. Local storage should be reserved for the text-based activities.

[ZCOEngineer]

Ok We shall fix and update

[sarithapillai8]

@carlan1 We have updated balloon risk calculation fixes in staging. QA testing is in progress.

[evinkalengaden]

We have updated the changes in Dashboard staging also noticed that, As we are rounding up decimal points, in some cases there might be a small difference in standard deviation and mean value. Please review.

[sarithapillai8]

@carlan1 {'type': 'manual_exit', 'value': true} added to temporal_slices on clicking back from games in dashboard-staging. Please review.

We have updated all the functionalities requested in this task.

[carlan1]

Reply acknowledged, will close issue when confirmed

[carlan1]

The balloon risk values seem accurate now.

Just to be clear, the breakpoint of each balloon should be a random draw from a gaussian/normal distribution defined by a custom mean and a custom standard deviation

Therefore, there should be some randomness in the sample mean and sample standard deviation for any sample of balloon breakpoints.

Please confirm this is the case / that this is the current implementation.

[sarithapillai8]

@carlan1 The current implementation is like this: We generate a list of random samples using the Gaussian method for the normal distribution. Then we adjust this set of values using the mean and standard deviation from activity settings. These values adjusted values are used as breakpoints for the game.

[carlan1]

What is the purpose of the adjustment? The mean and standard deviation should be used to define the Gaussian function used to generate the list of n samples

[sarithapillai8]

When we generate random n numbers, the standard deviation and mean using these values will not match the given values. So this list needs to be adjusted to match the given values. We need these random values between 1 and 128, that is why we have to do this adjustment. I have tried different ways and finally, this method worked. You can review the code here and can suggest if you have any alternative method: https://github.com/BIDMCDigitalPsychiatry/LAMP-activities/blob/master/BalloonRisk/src/components/BallonRisk/Balloon.tsx#L163

[carlan1]

Let us define the following: μ: The researcher-defined population mean σ: The researcher-defined population standard deviation x̄: sample mean s: sample standard deviation

From the code here, it looks like you're generating n samples from a uniform distribution, then shifting the values to force x̄ = μ, then uniformly adjusting the spread to force s = σ. The problem here is that the samples won't be normally distributed and there won't be randomness in s and x̄

Is it not possible to simply draw 15 samples from X ~ N(μ, σ^2)?

[carlan1]

Also please add {'type': 'manual_exit', 'value': False} when the game completes automatically (i.e. no manual exit)

[carlan1]

This code transforms uniform random values to normal distributions:

def random_gaussian(mean, std):
    u = rand.random();
    v = rand.random();
    if u == 0 or v == 0:
        return random_gaussian(mean, std)
    num = math.sqrt(-2.0 * math.log(u)) * math.cos(2.0 * math.pi * v)
    return (num * std) + mean;

EDIT: please see JS version below as per @avaidyam comment

[avaidyam]

@sarithapillai8 Please revisit the original discussion on gaussian vs. uniform distribution here, with provided code. (The code @carlan1 provided is Python and not JS. To ensure consistency across LAMP activities please reuse the same random_gaussian() code and ALWAYS avoid the use of Math.random().) You need to remove the > 0 && < 1 condition to re-center the mean:

function random_gaussian(mean, std) {
    let u = 0, v = 0;
    while(u === 0) u = Math.random();
    while(v === 0) v = Math.random();
    let num = Math.sqrt(-2.0 * Math.log(u)) * Math.cos(2.0 * Math.PI * v);
    return (num * std) + mean;
}

//cc @carlan1 @michaelmenon

[sarithapillai8]

@avaidyam @carlan1 We have already worked around this same method and we were not able to generate numbers those matching the provided mean and standard deviation. That is we tried different ways to find out the sample. We can change the method of random number generation in other games. But in this game, we need to restrict the values in the range 1-128. And we cannot have negative values.; That was why we used this condition: > 0 && < 1 Please see the samples using the above code :

Required values : Standard deviation - 37 , Mean - 64.5

120, 36, 23, 59, 106, 43, 116, 94, 7, 101, 87, 135, 55, 52, 26
Standard Deviation, σ: 38.837553418767
Mean, μ:    70.666666666667

44, 69, 105, -21, -35, 14, 98, 120, 132, 73, 53, -19, 47, 76, 91
Standard Deviation, σ: 50.298928638911
Mean, μ:    56.466666666667

-11, -1, 126, 88, 96, 71, 0, 106, 78, 114, 43, 58, 81, 160, 5
Standard Deviation, σ: 49.777906745865
Mean, μ:    67.6

[avaidyam]

The issue is that the > 0 && < 1 condition cannot be placed inside the random function. You must check the value and "re-roll the dice" outside of the random function. Furthermore it is not statistically correct to take a subsample of the random function and check its mean/stddev and expect it to be the same as the parent gaussian distribution, because it is a subsample. It is still possible to yield values that are three SD's below the mean (which we would consider as a statistical outlier).

[carlan1]

@avaidyam @carlan1 We have already worked around this same method and we were not able to generate numbers those matching the provided mean and standard deviation. That is we tried different ways to find out the sample. We can change the method of random number generation in other games. But in this game, we need to restrict the values in the range 1-128. And we cannot have negative values.; That was why we used this condition: > 0 && < 1 Please see the samples using the above code :

Required values : Standard deviation - 37 , Mean - 64.5

120, 36, 23, 59, 106, 43, 116, 94, 7, 101, 87, 135, 55, 52, 26
Standard Deviation, σ: 38.837553418767
Mean, μ:  70.666666666667

44, 69, 105, -21, -35, 14, 98, 120, 132, 73, 53, -19, 47, 76, 91
Standard Deviation, σ: 50.298928638911
Mean, μ:  56.466666666667

-11, -1, 126, 88, 96, 71, 0, 106, 78, 114, 43, 58, 81, 160, 5
Standard Deviation, σ: 49.777906745865
Mean, μ:  67.6

These samples look good to me with the exception of the negative values (and the zero)

Can we just add a line of code to check if the final value is less than or equal to zero? And if so, re-roll the dice as @avaidyam said

[sarithapillai8]

@avaidyam @carlan1 Please review the below code and let me know if we can do like this :

getRandomGaussian = function (mean: any, std: any) {
    let u = 0;
    let v = 0;
    while (u === 0) {
      u = Math.random();
    }
    while (v === 0) {
      v = Math.random();
    }
    let num = Math.sqrt(-2.0 * Math.log(u)) * Math.cos(2.0 * Math.PI * v);
    num = num * std + mean;
    if(num > 0 && num < 128) {
      return this.getRandomGaussian(mean, std)
    }
    return num
  };

  getRandomGaussianArray =  (mean: any, std: any): any => {
    const list = [];
    let i=0;
    const len = this.state?.balloon_count ?? 15
    for (i = 0; i < len; i++) {
      const num = this.getRandomGaussian(mean, std)
      list[i] = num;
    }
    this.setState({points: list})
    return list
  }

[avaidyam]

This is still incorrect. Please do this instead:

getRandomGaussian = function (mean: any, std: any) {
    let u = 0;
    let v = 0;
    while (u === 0) {
      u = Math.random();
    }
    while (v === 0) {
      v = Math.random();
    }
    let num = Math.sqrt(-2.0 * Math.log(u)) * Math.cos(2.0 * Math.PI * v);
    num = num * std + mean;
    return num
  };

  getRandomGaussianArray =  (mean: any, std: any): any => {
    const list = [];
    let i=0;
    const len = this.state?.balloon_count ?? 15
    for (i = 0; i < len; i++) {
      var num = 0
      while(num <= 0 || num >= 128) {
        var = this.getRandomGaussian(mean, std)
      }
      list[i] = num;
    }
    this.setState({points: list})
    return list
  }

[sarithapillai8]

Updated dashboard staging with balloon risk game changes and manual_exit - false added for all games. @carlan1

[carlan1]

The updated code seems nearly complete. There is only one potential issue which is

      let num = 0
      while(num <= 0 || num >= 128) {
        num = getRandomGaussian(mean, std)
      }
      list[i] = Math.round(num);

The line Math.round(num) will cause values < 0.5 to round to zero. To resolve this please change the while(num <= 0 || num >= 128) { line to while(num < 0.5 || num >= 128) {

// @avaidyam

[sarithapillai8]

@carlan1 Updated as you suggested.

[carlan1]

Looks good in staging