Week 2 Previous Work and Preliminary Approach
Previous Work
Previous studies to analyze animal behavior on bandit (or similar) tasks
- Ito & Doya, 2003 “Validation of Decision-Making Models and Analysis of Decision Variables in the Rat Basal Ganglia”
- Construct a variety of reinforcement learning models, compare them to one another and to markov models
- End with a reinforcement learning model with different learning rates for good and bad outcomes
- Lau & Glimcher, 2005 “Dynamic Response-by-Response Models of Matching Behavior in Rhesus Monkeys”
- Use regression models with regressors for previous choice, previous reward, interaction
- Show that trial-by-trial dynamics are important (as opposed to molar models)
- Show that future choices can depend on past choices
- Sugrue, Corrado, Newsome, 2005 “Linear-Nonlinear-Poisson Models of Primate Choice Dynamics”
- Use an LNP model with reward-by-choice interaction as the predictor
- Show that trial-by-trial dynamics are important (again - this was a big deal at the time)
- Argue for hyperbolic, rather than exponential, discounting
- More recent work
- Typically assumes a TD learning process a la Ito & Doya (or even simpler)
- Looks for neural correlates, mechanisms
Preliminary Analysis
- Approach: Build regression models a la Lau & Glimcher, compare them to Markov models, fancier approaches
- Ultimate goal: construct a reinforcement learning model which explains data as well as fancy approaches
- For today: Markov models & regression models, an idea for a fancier approach
- Model Comparison: Cross-validate over sessions (for now: even/odd). Compute “Normalized likelihood”
Markov Models
Logistic Regression Models
Logistic Regression Fit Example
Model Comparison
Written on October 5, 2015