step type in a staircase

[kschuler]

[kcm1231]

Maybe on Friday we can look at some data files from the RAs runs and make sure the staircase is adjusting accuracy the way we want?

[kcm1231]

Once the code is set and I'm able to have RAs run through the entire experiment without errors, I'll need to make sure the accuracy is adjusting appropriately. If not, it's possible we may need to either 1) change the steptype, or 2) change the way the staircase is working and use pilot data to calculate specific parameters...

[kcm1231]

So, I just did a quick check on the longest run that we've gotten so far, and I looked at the accuracy in the Environmental condition. For the "easy" staircase, the RA only got 18/60 trials correct. For the "hard" staircase, she got 31/60 correct. So obviously something isn't working correctly. Will see what I can figure out...

[kschuler]

It could be that they are just equally hard.

On Thu, Feb 2, 2017 at 3:27 PM, kcm1231 notifications@github.com wrote:

So, I just did a quick check on the longest run that we've gotten so far, and I looked at the accuracy in the Environmental condition. For the "easy" staircase, the RA only got 18/60 trials correct. For the "hard" staircase, she got 31/60 correct. So obviously something isn't working correctly. Will see what I can figure out...

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/kschuler/words-in-noise-EEG/issues/18#issuecomment-277073343, or mute the thread https://github.com/notifications/unsubscribe-auth/AF0T7N-_QBLH73CIKnPSp679nVANRAEjks5rYjw0gaJpZM4LiJjB .

[kcm1231]

I think that I figured out what was happening here. Traditional staircase procedures are used to discover a threshold (ie a JND), and they also usually change the level of the noise rather than the signal. With our staircase, we want to keep them at a predetermined level, and we are changing the value of the signal rather than the noise. So, this means that instead of using the 3up 1down and 1up 1down, we need to reverse the step directions, and probably also adjust the values. I'm going to run some tests using a 1up 5down for easy and a 1up 2down for hard. We'll see how that works out.

After talking with Peter, we may try to implement a different check on accuracy for the main staircase. The baseline staircase will hopefully get them closer to the desired easy and hard thresholds, and then in the main staircase, we want to implement a piece of code that checks the accuracy using a sliding window (i.e. rolling average) of the previous 10 trials, then adjusts either up or down depending on if accuracy is below or above the desired threshold. Our new postdoc, Andrew, has a bunch of coding experience but hasn't really used Python, but he said that if you're busy working on the other edits and your own stuff, he can try to implement this accuracy check for us. However, let me know if this is clear, and if it sounds like something that would be really quick for you to add in. If it's an easy change, I don't want him to mess with your code since you have it very well organized and formatted...

[kschuler]

It is easier for me to edit later if people haven't added things... but if you need it asap that is fine.

Checking the accuracy isn't going to be straightforward because there are several interleaved staircases... it isn't hard to do, you just have to make sure that you are checking the previous trials in the correct staircase and level.

On Fri, Feb 3, 2017 at 10:24 AM, kcm1231 notifications@github.com wrote:

I think that I figured out what was happening here. Traditional staircase procedures are used to discover a threshold (ie a JND), and they also usually change the level of the noise rather than the signal. With our staircase, we want to keep them at a predetermined level, and we are changing the value of the signal rather than the noise. So, this means that instead of using the 3up 1down and 1up 1down, we need to reverse the step directions, and probably also adjust the values. I'm going to run some tests using a 1up 5down for easy and a 1up 2down for hard. We'll see how that works out.

After talking with Peter, we may try to implement a different check on accuracy for the main staircase. The baseline staircase will hopefully get them closer to the desired easy and hard thresholds, and then in the main staircase, we want to implement a piece of code that checks the accuracy using a sliding window (i.e. rolling average) of the previous 10 trials, then adjusts either up or down depending on if accuracy is below or above the desired threshold. Our new postdoc, Andrew, has a bunch of coding experience but hasn't really used Python, but he said that if you're busy working on the other edits and your own stuff, he can try to implement this accuracy check for us. However, let me know if this is clear, and if it sounds like something that would be really quick for you to add in. If it's an easy change, I don't want him to mess with your code since you have it very well organized and formatted...

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/kschuler/words-in-noise-EEG/issues/18#issuecomment-277275341, or mute the thread https://github.com/notifications/unsubscribe-auth/AF0T7CnqS0Xp4FkLvjH91JCwo5AQlTUkks5rY0aTgaJpZM4LiJjB .

[kcm1231]

Ok! Yeah, I figured it would be better to have you do it yourself, but wanted to make sure. I'll tell him not to worry about it. We don't need that any sooner than any of the other edits we've already talked about, just wanted to make sure that wasn't going to add a huge chunk of time onto what you already had to do!

[kcm1231]

Hey there,

Experimenting with some different nUP and nDOWN numbers, and on the last couple of runs it's been doing something weird. I'll send the parameter and exp file used, but I had it set at an 1up 10down for easy and 2up 3down for hard, with a step size of 6 for the baseline and 2 for the main staircase. I'm looking at the intensity levels of the stimuli, and on the easy baseline staircase instead of changing by a step size of 6, on one trial the intensity jumps from 7 to 15, and then on another trial from 15 to 31, and then from 31 to 63 (meaning the sounds are super loud and distorted). The step type is db, not log. I have no clue why this could be happening. Rami_1_A_AV-2017-Feb-07-115207.xlsx

experiment_Rae_edit.txt

[kschuler]

If you just want it to step up by the step size (6), then you need it to be 'lin' not 'db'.

On Tue, Feb 7, 2017 at 1:22 PM, kcm1231 notifications@github.com wrote:

Hey there,

Experimenting with some different nUP and nDOWN numbers, and on the last couple of runs it's been doing something weird. I'll send the parameter and exp file used, but I had it set at an 1up 10down for easy and 2up 3down for hard, with a step size of 6 for the baseline and 2 for the main staircase. I'm looking at the intensity levels of the stimuli, and on the easy baseline staircase instead of changing by a step size of 6, on one trial the intensity jumps from 7 to 15, and then on another trial from 15 to 31, and then from 31 to 63 (meaning the sounds are super loud and distorted). The step type is db, not log. I have no clue why this could be happening. Rami_1_A_AV-2017-Feb-07-115207.xlsx https://github.com/kschuler/words-in-noise-EEG/files/758599/Rami_1_A_AV-2017-Feb-07-115207.xlsx

experiment_Rae_edit.txt https://github.com/kschuler/words-in-noise-EEG/files/758601/experiment_Rae_edit.txt

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/kschuler/words-in-noise-EEG/issues/18#issuecomment-278093084, or mute the thread https://github.com/notifications/unsubscribe-auth/AF0T7FfHc3SWz5HRflpBqIhDSINFU1M8ks5raLZhgaJpZM4LiJjB .