What is RL?

[falcondai]

• Compare with an adjacent settings: planning.
  • Bellman equation
• Informational ignorance (per Kakade’s thesis)
• Inference in language models

[falcondai]

What makes RL exciting for me?

• as a model to study science, the experimentation and the exploration.
• as a model to inform rational decision making, not just for machines, it should tell us how to act in life. That is also why I have a preference for formal results. I like interesting questions and new ways to think.

[falcondai]

• unknown parameters
  • c.f., bandits with known parameters
  • c.f., planning, search. Robotics, game playing
• bandit feedback, c.f., expert problems (Avrim’s traffic example)
• stochastic and adversarial