Spring 2026 · College of William & Mary
Decision-Making under Uncertainty
CSCI 680-03 · Special Topics
Many of the applications that matter most for artificial intelligence involve deciding what to do when you cannot be sure what will happen next: autonomous driving, disaster response, transportation optimization, healthcare. Deciding well in these settings means balancing several objectives at once, reducing risk, and taking seriously the different sources of uncertainty involved. This course develops the principled computational techniques for doing that: single-shot optimization, probabilistic reasoning, and sequential decision-making. We also look at decision-making for societal impact and at how fairness and equity can be modelled mathematically.
- Instructor
- Ayan Mukhopadhyay · contact
- Lectures
- Tuesdays & Thursdays, 5:00–6:20 PM
- Location
- Integrated Science Center 3280
- Office
- Room 1392, Integrated Science Center IV
- Office hours
- Wednesdays 3:00–4:00 PM and Fridays 12:00–2:00 PM
If you cannot make it, email me and we will find another time. - Level
- Undergraduate / graduate
Prerequisites
Previous experience with Python is strongly recommended. Beyond that:
- Graduate students: a fundamental course in statistics and one in algorithms are mandatory. A prior course in artificial intelligence is helpful but not required.
- Undergraduates: fundamental courses in algorithms and statistics.
Course material
We follow two textbooks, plus additional material posted to the course site.
- Algorithms for Decision Making, Kochenderfer, Wheeler & Wray. Free PDF, or buy it from MIT Press.
- Artificial Intelligence: A Modern Approach, 4th US edition, Russell & Norvig. Available at the campus bookstore, and on reserve at the library.
We will also read chapters from Thinking, Fast and Slow (Kahneman) and The Ethical Algorithm (Kearns & Roth), among others. You do not need to buy these; copies of the relevant chapters will be posted. Expect a good deal of assigned reading besides, from technical papers to popular science articles; these often drive class discussion, and pop quizzes.
For the introductory probability and statistics classes we work through the mathematics on the board, and concise PDFs go up on Blackboard afterwards. You are still encouraged to take your own notes; they will help for pop quizzes and the mid-term. Lectures on decision-making, after the introductory material, are mostly on slides, which are also posted.
Grading
- Homework assignments35%
- Mid-term25%
- Project / end-term20%
- Pop quizzes10%
- Paper presentation5%
- Class participation5%
A note on slides.
I do not post lecture slides publicly. If you are teaching a similar course and would find them useful, get in touch with a sentence about your course and I will happily share them. Enrolled students get everything through Blackboard.
Schedule
Fifteen weeks. The plan may shift depending on where discussions in class take us; exact dates for each session, and all deadlines, are on Blackboard.
| Week | Topics |
|---|---|
| Week 1 | Introduction and background |
| Week 2 | Foundations of probability and statisticsHW 0 |
| Week 3 | Foundations of probability and statistics Probabilistic representation |
| Week 4 | Probabilistic representation Probabilistic inferenceHW 1 One session replaced by an invited talk over Zoom. |
| Week 5 | Parameter learning Making simple decisions: utility theory and prospect theory |
| Week 6 | Sequential problems: Markov decision processes MDPs: exact solution methods |
| Week 7 | Offline planning Online planning: sampling-based search methodsHW 2 Additional office hours for mid-term review. |
| Week 8 | Spring break: no classHW 3 HW 3 released; due after the mid-term. |
| Week 9 | Mid-term (in class only) Mid-term review |
| Week 10 | Policy-based methods Semi-Markov and continuous-time Markov decision processes |
| Week 11 | State uncertainty and POMDPs Introduction to reinforcement learning |
| Week 12 | Combining learning and online planning: AlphaZero Introduction to mathematical programmingHW 4 |
| Week 13 | Model predictive control Decision-making under non-stationarity |
| Week 14 | Algorithmic fairness in optimization and decision-making Decision-making for social impactHW 5 |
| Week 15 | Paper presentationsProject Project due at the end of the exam period. |
Assignments and policies
Homework
Assignments mix programming with analytical and mathematical problems. All programming is in Python. If you have no Python background we will spend some time on it, and assignment 0, handed out in the first week, exists to get you started. Unless stated otherwise, homework is released before midnight on Tuesdays and due the following Tuesday at 11:59 PM.
Late policy
Late submissions decay exponentially. Your score is S · e−t/2100, where t is the delay in minutes and S is the score you earned. So on an 80: two minutes late is still 80; one day late is 40; two days late is 10.
Submissions
Analytical solutions, programming assignments, and project reports all go through Blackboard. I encourage you to use LaTeX for the analytical work; a sample format is on Blackboard, and Overleaf makes it easy to edit without installing anything.
Large language models
You may use LLMs to generate project ideas, improve your writing, make study materials, and clarify assignments. You may not use them to summarise papers, produce critical reviews of papers, or help with programming on homework.
Academic integrity
I encourage you to work with other students in discussing papers and class content. But unless stated otherwise, submissions must be your own individual effort, completed independently. Anything else is a violation of academic integrity.
Attendance
Attendance is required, and pop quizzes only happen in class. That said, if you are unwell, do not worry about class. Take care of yourself, let me know, and we will arrange for you to make up what you missed.
Add/drop, withdrawal, and reading-period dates follow the William & Mary academic calendar. There is no final exam for this course.