Week 2 Previous Work and Preliminary Approach

Previous Work

Previous studies to analyze animal behavior on bandit (or similar) tasks

  • Ito & Doya, 2003 “Validation of Decision-Making Models and Analysis of Decision Variables in the Rat Basal Ganglia”
    • Construct a variety of reinforcement learning models, compare them to one another and to markov models
    • End with a reinforcement learning model with different learning rates for good and bad outcomes
  • Lau & Glimcher, 2005 “Dynamic Response-by-Response Models of Matching Behavior in Rhesus Monkeys”
    • Use regression models with regressors for previous choice, previous reward, interaction
    • Show that trial-by-trial dynamics are important (as opposed to molar models)
    • Show that future choices can depend on past choices
  • Sugrue, Corrado, Newsome, 2005 “Linear-Nonlinear-Poisson Models of Primate Choice Dynamics”
    • Use an LNP model with reward-by-choice interaction as the predictor
    • Show that trial-by-trial dynamics are important (again - this was a big deal at the time)
    • Argue for hyperbolic, rather than exponential, discounting
  • More recent work
    • Typically assumes a TD learning process a la Ito & Doya (or even simpler)
    • Looks for neural correlates, mechanisms

Preliminary Analysis

  • Approach: Build regression models a la Lau & Glimcher, compare them to Markov models, fancier approaches
  • Ultimate goal: construct a reinforcement learning model which explains data as well as fancy approaches
  • For today: Markov models & regression models, an idea for a fancier approach
  • Model Comparison: Cross-validate over sessions (for now: even/odd). Compute “Normalized likelihood”

Markov Models

Markov Models

Logistic Regression Models

Regression Models

Logistic Regression Fit Example

Regression Models

Model Comparison

Model Comparison

Written on October 5, 2015