search

int-brain-lab / iblenv

Unified environment and Issue tracker for all IBL
MIT License
10 stars 10 forks source link

[Bug report] - compute_bias_info returns different bias on each call #174

Open cmlangdon opened 3 years ago

cmlangdon commented 3 years ago

Describe the bug 'training.compute_bias_info' returns different values for 'ps20' and 'ps80' on each call for some eids.

To Reproduce Steps to reproduce the behavior:

  1. eid = 'bd456d8f-d36e-434a-8051-ff3997253802'
  2. trials_all = one.load_object(eid, 'trials')
  3. trials = dict()
  4. trials['temp_key'] = trials_all
  5. perf_easy, n_trials, ps20, ps80, rt = training.compute_bias_info(trials, trials_all)
nbonacchi commented 3 years ago

sorry where is training.compute_bias_info coming from?

cmlangdon commented 3 years ago

brainbox.behavior

nbonacchi commented 3 years ago

hummm I don't see any probabilistic calls in either compute_bias_info or compute_psychometric. From the docstring it looks like compute_bias_info requires 3 training sessions, maybe it's that? """ Compute all relevant performance metrics for when subject is on biasedChoiceWorld

:param trials: dict containing trials objects from three consective training sessions,
keys are session dates
:type trials: Bunch
:param trials_all: trials object with data concatenated over three training sessions
:type trials_all: Bunch
:returns:
    - perf_easy - performance of easy trials for each session
    - n_trials - number of trials in each session
    - psych_20 - parameters for psychometric curve fit to trials in 20 block over all sessions
    - psych_80 - parameters for psychometric curve fit to trials in 80 block over all sessions
    - rt - median reaction time for zero contrast stimuli over all sessions
"""
cmlangdon commented 3 years ago

I suppose it could be coming from local minima in the optimization in 'compute_psychometric'. I'll look a little more closely at it. Maybe using more trials from three sessions leads to unique solutions.

k1o0 commented 3 years ago

The variation is coming from mle_fit_psycho. The parameter bounds are very liberal and the number of fits is just 5, meaning the parameters sometimes don't converge. Next theory group meeting we will moot changing these parameters.