neuroailab / tdw_physics

Generic structures to create TDW physics datasets
5 stars 5 forks source link

metrics for curio-X series of projects #1

Open yamins81 opened 3 years ago

yamins81 commented 3 years ago
  1. define the metric Metric

  2. compute for pairs of conditios

m1 = Metric(set of trajectories for condition 1 [object1], also for set of alternatives) m2 = Metric(set of trajectories for condition 2 [object1])

  1. compute linking function

linking_function(m1, m2) = probability of choice of m1

linking_function ~ softmax ??

  1. sample draw sample from the distrubiton of choices across a bunch of conditions to get trials

  2. correlate then compare model's trials to real huan trials and divide by squareroot of product of reliabilies for model and human

========but what specific set of metrics?

-- total variance in the velocity vector of object just after first collision

-- proportion of trials in which the dropped object comes to a steady collision state (e.g. suppporting)

-- some metric of the sharpness of transition in the response function of variability in velocity vector or positional dispersion as a function of initial state variability of dropped object

-- some metric of the sharpness of transition in the response function of variability in velocity vector or positional dispersion as a function of initial pose of the dropped object

-- maximum velocity of (dropped or target?) obtained during trajectory

-- net displacement of target

-- specificity of some feature above to pairing of objects e.g. for some pairs of obejcts, there will be especially more of a given feature relative to other pairs;

mcfrank commented 3 years ago
  1. whether there is a big effect (summed velocity after collision)
  2. uncertainty in the size of effect (variance V after collison)
  3. interesting outcomes (proportion support/contain)
  4. skills-y-ness of the task (proposal: loss on linear function predicting final state from initial state)
  5. made the target move (summed velocity of target)
  6. uniqueness of outcome (can only get this effect for a particular pair)