Literature

Study material

Lecture material (mandatory):

• Deep Reinforcement Learning, Aske Plaat, 2023 (free pdf).

  • Main course textbook. 

• Reasoning with LLM, a survey, Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, Thomas Back, arXiv 2407.11511, 2024

  • Paper for last lecture on RL+LLM.

Prior knowledge & Experimentation (mandatory): 

• Probability (interactive notebook)

  • You should be familiar with the basic concepts of probability (for any machine learning course). This should have appeared in previous courses, but otherwise the above notebook gives a summary. 

• Reports: Advice and Common Errors. (reader)

  • To properly do experiments and write a report, take notice of the advice in the above document. 

Additional resources (recommended): 

• Policy Based Reinforcement Learning, Thomas Moerland, 2021 (course notes).

  • The appendix is optional, but may help to see RL in the bigger picture of machine learning. 

• Reinforcement Learning: An Introduction, Sutton and Barto. (2nd edition, 2020, pdf) 

  • The textbook by Sutton and Barto is arguably the 'Bible of Reinforcement Learning', for all fundamental (tabular) concepts. 

  • For coverage of all fundamental (tabular) algorithms, read Chapters 1-8, 12-12.2 and 13. 

  • You could consider the first edition of the book as well, since this mostly covered these basics (the 2nd edition is more extensive). 

• Learning to Play, Aske Plaat, 2020. 

  • A previous version of this course was based on this book. It also covers classical search algorithms, while the current course book focuses purely on learning methods. 

Exam training (recommended): 

• EduGym contains a range of interactive notebooks that explain specific concepts of reinforcement learning

• This notebook on Markov Decision Process & Dynamic Programming gives code illustration of MDP definitions and Dynamic Programming

Experimentation

• Reports: Advice and Common Errors

  • This document explains how to structure your course reports, and how to run experiments (incl. hyperparameter optimization). See Assignments.

• Running on GPU