search

Rohan2821999 / MathCog_Modelling

A cognitive model to understand basic math acuity in children using weber fraction and numerical distance effect based algorithms
1 stars 0 forks source link

Correlation between simulated and actual data #2

Closed Rohan2821999 closed 7 years ago

Rohan2821999 commented 8 years ago

@cbattista Below is the graph for the correlation between actual easiness vals and simulated vals over all the points. Looks pretty bad :(

fig

Here is what weber fraction, m, intercept values I have used for the ages : 

if (18 >= age_grade >= 15):
            Child = Person(0.16,-16,650)
elif(14 >= age_grade >= 11 ):
            Child = Person(0.22,-25,750)
elif(10 >= age_grade >= 8):
            Child = Person(0.25,-30,900)
elif(age_grade == 7):
            Child = Person(0.27,-32,1000)
# where age_grade is the age of the person
Rohan2821999 commented 8 years ago

And the Pearson Coefficient Matrix is 

[[ 1. 0       0.01059]
[0.01059       1.0]]

So the correlation is almost 0

cbattista commented 8 years ago

Well it looks like it works pretty well (top right cluster) for some and not for others (the rest of it).

So now, we can think about why that would be...firstly, I realized I didn't pull out the 'practice' trials out of the data so that might have something to do with it...so that's one thing. But it's kind of interesting to see that there seem to be 'categories' there...

So one thing we want to ask is, do we have 'types' of people, where it fits for some and not for others? Or, is it just a huge mess?

On Mon, Aug 22, 2016 at 1:01 PM, Rohan Hundia notifications@github.com wrote:

@cbattista https://github.com/cbattista Below is the graph for the correlation between actual easiness vals and simulated vals over all the points. Looks pretty bad :(

[image: fig] https://cloud.githubusercontent.com/assets/13100688/17869396/f7749f1e-6867-11e6-8dc5-234b8a2e0472.JPG

Here is what weber fraction, m, intercept values I have used for the ages :

if (18 >= age_grade >= 15): Child = Person(0.16,-16,650) elif(14 >= age_grade >= 11 ): Child = Person(0.22,-25,750) elif(10 >= age_grade >= 8): Child = Person(0.25,-30,900) elif(age_grade == 7): Child = Person(0.27,-32,1000)

where age_grade is the age of the person

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Rohan2821999/MathCog_Modelling/issues/2, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOuO4EE6bS-0rvJOqp4smK97A4VRkXFks5qigAigaJpZM4JqPwO .

Rohan2821999 commented 8 years ago

Checked that.. all the 4 clusters exist and are quite prominent in all 'types' of people. So the clusters are not independent to 'categories' or 'types' of people.

cbattista commented 8 years ago

Ok, good to know. So now let me see about removing all those practice trials...

On Mon, Aug 22, 2016 at 1:48 PM, Rohan Hundia notifications@github.com wrote:

Checked that.. all the 4 clusters exist and are quite prominent in all 'types' of people. So the clusters are not independent to 'categories' or 'types' of people.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Rohan2821999/MathCog_Modelling/issues/2#issuecomment-241545309, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOuO-wtZaZVcXr9_-s6IWFCQ_iH7G2Rks5qigsQgaJpZM4JqPwO .

Rohan2821999 commented 8 years ago

NDE Graph of Simulated Easiness Values vs Actual Easiness Values for all correct trials:

image

Following slope and intercept values were used for different age ranges:

Age 15-18: slope = -16, intercept = 750
Age 11-14: slope = -30, intercept = 950
Age 10-8: slope = -40, intercept = 1200
Age 7: slope = -50, intercept = 1500
cbattista commented 8 years ago

Hmm quite a blob we have there...

A few questions:

Can you just plot RTs (instead of easiness) Can you color code the different age ranges - curious to see whether data fits better for any of the age ranges... What's up with the 'stripe' pattern in the simulated data?

On Tue, Aug 23, 2016 at 2:41 PM, Rohan Hundia notifications@github.com wrote:

NDE Graph of Simulated Easiness Values vs Actual Easiness Values for all correct trials:

[image: image] https://cloud.githubusercontent.com/assets/13100688/17910744/1a26f598-693f-11e6-91bb-8468097d3cc3.png

Following slope and intercept values were used for different age ranges:

Age 15-18: slope = -16, intercept = 750 Age 11-14: slope = -30, intercept = 950 Age 10-8: slope = -40, intercept = 1200 Age 7: slope = -50, intercept = 1500

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Rohan2821999/MathCog_Modelling/issues/2#issuecomment-241889170, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOuO7wNJmg4KvuoI9pW5xjTOjN3hjXBks5qi2kbgaJpZM4JqPwO .

Rohan2821999 commented 8 years ago

Sure,So you want a graph of the Simulated RT's vs Actual RT's color coded according to ages, right? Or do you want a graph of simulated RT's vs distance (color coded)?

Not surely about the stripe pattern, would look into it..

cbattista commented 8 years ago

Yes, simulated RTs vs actual RTs color coded according to ages. for example... Age 15-18: black Age 11-14: blue Age 10-8: green Age 7: red

On Tue, Aug 23, 2016 at 3:35 PM, Rohan Hundia notifications@github.com wrote:

Sure,So you want a graph of the Simulated RT's vs Actual RT's color coded according to ages, right? Or do you want a graph of simulated RT's vs distance (color coded)?

Not surely about the stripe pattern, would look into it..

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Rohan2821999/MathCog_Modelling/issues/2#issuecomment-241902003, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOuOw7WK8VWeLDvGn0h5OM4QWlQt1wIks5qi3XLgaJpZM4JqPwO .

Rohan2821999 commented 8 years ago

Here is the graph:

image

Age 15-18: red Age 11-14: blue Age 8-10: green Age 7: brown

cbattista commented 8 years ago

So for the 7 year-olds there might be a bit of a (weak) relationship there but the rest is indeed quite bad....definitely need a better model!

Rohan2821999 commented 8 years ago

I have added two graphs (Actual RT vs Ratios and Sim_RT vs Ratios) ::: Red - Acc = 0 , Green - Acc = 1

image

image

cbattista commented 8 years ago

neat - can you make the circles more transparent (using the 'alpha' argument in the scatter function) it will make things easier to see...

On Thu, Aug 25, 2016 at 2:32 PM, Rohan Hundia notifications@github.com wrote:

I have added two graphs (Actual RT vs Ratios and Sim_RT vs Ratios) ::: Red

  • Acc = 0 , Green - Acc = 1

[image: image] https://cloud.githubusercontent.com/assets/13100688/17986581/3f310210-6ad0-11e6-9cb9-5a3b419f2f64.png

[image: image] https://cloud.githubusercontent.com/assets/13100688/17986660/a8d7f642-6ad0-11e6-884a-fcff50518b10.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Rohan2821999/MathCog_Modelling/issues/2#issuecomment-242547839, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOuO1lx8NQcNQ4H71pu-Haw9gLJ6RAGks5qjgncgaJpZM4JqPwO .

Rohan2821999 commented 8 years ago

image

image

Rohan2821999 commented 8 years ago

Graph of Avg_RT Values (Actual) vs Avg Accuracy values for each subject:

image

Doesn't quite look as expected..

Rohan2821999 commented 8 years ago

Plot of Simulated RT (sampled from normal distribution with std = intercept) vs Ratios. The plot more or less matches the Actual RT vs Ratio scatter plot..

image

Should I re-plot my easiness values graph now (probably could get better results)?

cbattista commented 8 years ago

yeah, let's try that...

On Fri, Aug 26, 2016 at 12:18 PM, Rohan Hundia notifications@github.com wrote:

Plot of Simulated RT (sampled from normal distribution with std = intercept) vs Ratios. The plot more or less matches the Actual RT vs Ratio scatter plot..

[image: image] https://cloud.githubusercontent.com/assets/13100688/18017584/f821bf92-6b86-11e6-9461-10ebbd3949c2.png

Should I re-plot my easiness values graph now (probably could get better results)?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Rohan2821999/MathCog_Modelling/issues/2#issuecomment-242827058, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOuO6DTwMEglrs1w7YAJFXYj6TrYAYbks5qjzwTgaJpZM4JqPwO .

Rohan2821999 commented 8 years ago

RT Histogram Chart

image

Rohan2821999 commented 8 years ago

Histogram Plot for Simulated and Actual E

image

Rohan2821999 commented 8 years ago

for w = 0.17

image

Rohan2821999 commented 8 years ago

Kinda weird still overestimating difficulty a bit at a pretty low w..

cbattista commented 8 years ago

huh, I guess it's the NDE driving the RT there then...

On Fri, Aug 26, 2016 at 3:26 PM, Rohan Hundia notifications@github.com wrote:

Kinda weird still overestimating difficulty a bit at a pretty low w..

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Rohan2821999/MathCog_Modelling/issues/2#issuecomment-242866809, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOuO3ZjFvWGvzyAf_Sdzl4SGNEy2N2Lks5qj2gmgaJpZM4JqPwO .