Syllabus — Fall 2026

← Back to course home · Schedule · Project · Policies

Below is the week-by-week plan, including the readings assigned for each week. Please read and be prepared to discuss the required readings before the specified class session.

Most of these topics can be (and often are) the focus of entire courses. We'll generally only scratch the surface, but hopefully inspire you to go deeper in areas that interest you — you'll find plenty of open research questions in each. Optional readings are listed for most sessions for students who want to delve deeper, and often provide useful context for related project work.

Readings marked 🔒 require CMU credentials or an institutional subscription. Readings with a PDF in this repo are linked directly.

Week 1 — Introduction and project scoping

Aug 25, 27, 28

Tuesday: Intro and overview

An introduction to the class, its goals, and an overview of the project options to help you decide what you're interested in working on for the semester.

Thursday: Scoping, problem definition, and balancing goals

Well before the outset of technical work, a decision needs to be made about whether a given policy problem can and should be addressed with machine learning: is the problem significant, feasible to solve with a technical approach, and of sufficient importance to policymakers that they will devote resources to implementing the solution? How will success be measured? How will the often-competing goals of efficiency, effectiveness, and equity be balanced?

Required (Thursday):

Optional:

Friday lab: Tech session — basic setup

We'll make sure everyone can connect to the server over ssh, has GitHub access, and can reach the database from both psql and DBeaver.

Week 2 — Case studies and acquiring data

Sep 1, 3, 4

This week we'll organize groups and begin project work.

Tuesday: Case studies

Practical examples are a great way to understand the nuance of applying machine learning to policy problems. Tuesday is a class discussion of a recent application, scoping the case together in breakout groups.

Required (Tuesday):

Thursday: Acquiring data, privacy, and record linkage

Acquiring data from a project partner is often an involved process with legal and technical dimensions. Researchers need to understand how the data may and may not be used (typically formalized in a data use agreement as well as underlying law) and ensure the privacy of individuals in the dataset is protected, through both access restrictions and techniques like anonymization. Once acquired, data often needs to be transformed for ingest, linked across sources, and structured for analysis. We'll also spend some time on working effectively with your project team.

Optional:

Friday lab: Tech session — remote workflows

A workshop on the remote workflow tools you'll use for your project.

Week 3 — Data exploration and teamwork

Sep 8, 10, 11

Project work this week should include continuing to develop and refine your scope as you begin to explore the data.

Tuesday: Data exploration

An overview of exploratory data analysis, particularly with downstream ML in mind. Data exploration is fundamental to understanding the nuances of the data and how the policy problem you initially scoped can be formulated as a machine learning problem. This involves generating and plotting summary statistics, exploring trends over time, understanding rapid changes in distributions, and identifying missing data and outliers. Typically it should involve considerable input from domain experts as you develop an understanding of how the data relates to the underlying generative process, as well as its idiosyncrasies and limitations.

We'll also set aside about 30 minutes on Tuesday for you to meet with your project team and discuss your scope.

Thursday: Project work — data exploration, scoping, and proposal planning.

Friday lab: Tech session — git and GitHub

A tutorial and refresher on using GitHub for your project.

Week 4 — Analytical formulation, baselines, and ML pipelines

Sep 15, 17, 18

📌 Due Tuesday, Sep 15: project proposal with scope and descriptive statistics

Tuesday: Analytical formulation and baselines

Distinct from initial scoping, a true analytical formulation of your policy problem can only come after you understand the data at hand — which in turn usually produces a better understanding of the problem itself. Here you'll ask how specifically your target variable (if relevant) is defined in the data, what types of information are available as predictors, and what baseline you'll measure performance against. Very rarely is the appropriate baseline as simple as random choice or the population prevalence. Rather, it should reflect what would be expected to happen otherwise: perhaps a simple decision rule an expert would come up with, or a pre-existing statistical model the current effort seeks to replace.

Required (Tuesday):

Optional:

Thursday: Building ML pipelines

End-to-end ML pipelines can quickly become unwieldy with several moving pieces, and well-structured, modular code is often critical to detecting and fixing bugs. This session gives an overview of the pipeline, each underlying element, and some best practices for building them.

Optional:

Friday lab: Tech session — Python and SQL

Week 5 — Choosing performance metrics & evaluating classifiers, part 1

Sep 22, 24, 25

📌 Due Tuesday, Sep 22: peer reviews of three project proposals

Pipeline development should continue this week, with a focus on producing the simplest possible version of the full system.

Tuesday: Choosing metrics

In most cases, a vast array of methods — each with tunable hyperparameters — can be brought to bear on your modeling question. How do you decide which models are better than others, and how can you be confident that decision carries forward when the model is deployed? How should you balance performance and fairness in these decisions? Are models performing similarly well giving similar predictions, and what should you do if they aren't? We start with the choice of performance metrics.

Required (Tuesday):

Optional:

Thursday: Project work

Friday lab: Tech session — triage configuration

An introduction to triage, the ML pipeline toolkit we use for the class project. See also the triage documentation.

Week 6 — Choosing performance metrics & evaluating classifiers, part 2

Sep 29, Oct 1, 2

By this week, your group should have a very simple version of an end-to-end pipeline with preliminary results for a single model specification.

Tuesday: Model selection and validation

Continuing from last week, we focus on validation strategies that reflect how you want your model to generalize — in particular, the common case of modeling contexts with a strong temporal component where predicting into the future is the goal, and how your choice of training and validation sets should reflect that.

Required (Tuesday):

Optional:

Thursday: Temporal validation deep dive — we meet together as a class to work through temporal validation using a few class project examples.

Friday lab: Check-ins begin. Regular team check-ins start this week to give feedback on project progress.

Week 7 — Feature engineering and imputation

Oct 6, 8, 9

In many real-world contexts, expressing domain expertise through thoughtful feature engineering can dramatically improve model performance by capturing what underlying factors are likely to be predictive and helping the model find these relationships. Likewise, most datasets you'll encounter in practice are littered with outliers, inconsistencies, and missingness, and handling these in a smart way can be critical to a project's success.

Tuesday: Feature engineering and imputation

Required (Tuesday):

Optional:

Thursday: Project work

Friday lab: Check-ins

Fall break — no classes

Oct 13, 15, 16

Week 8 — Feature engineering in triage

Oct 20, 22, 23

📌 Due Wednesday, Oct 21: modeling plan and temporal validation configuration

This week we get concrete about translating the feature ideas from week 7 into triage configuration: how features are specified, how aggregations over time and entity are defined, and how imputation is handled in the pipeline.

Tuesday: Features and imputation slides and the triage features cookbook

Thursday: Triage office hours and Q&A

Friday lab: Check-ins

Week 9 — ML modeling in practice

Oct 27, 29, 30

📌 Due Wednesday, Oct 28: V0 baseline results and planned feature list

Pipeline development and refinement should continue this week with a widening set of model specifications and features to explore.

Tuesday: ML modeling in practice

Practical guidance on building ML models for real-world projects: how should you think about what types of models to build? What hyperparameters should you explore, and how do you design a hyperparameter grid?

Required (Tuesday):

Optional:

Thursday: Project work

Friday lab: Check-ins

Week 10 — Choosing performance metrics & evaluating classifiers, part 3

Nov 3, 5, 6

📌 Due Wednesday, Nov 4: V0 modeling results — updated validation splits, features, and baseline results.

Tuesday: No class (Election Day)

Thursday: Model selection, part 3

We return to model selection, digging into how to winnow a large number of model specifications down to one or a handful that perform "best" for some definition of best. We focus on the common case of ML problems with a strong time-series component and the desire to balance performance and stability in model selection.

Friday lab: Check-ins

Week 11 — Model interpretability and ethics

Nov 10, 12, 13

📌 Due Wednesday, Nov 11: weekly project update

By this week, project work should begin to focus more heavily on evaluation, model selection, and interpretation.

Tuesday: Model interpretability

Model interpretability can be thought of at two levels: global (how the model works in aggregate) and local (why an individual prediction came out as it did). We focus on practical aspects and applications at both levels: understanding how a model performs globally, what it means to compare that performance across specifications, how these methods help researchers debug and improve models, how they build trust among stakeholders (including a growing legal movement toward a "right to explanation"), how they help those acting on predictions know when to override the model with their judgment, and — importantly — how they help decide not only whom to intervene on but what intervention to take.

Required (Tuesday):

Optional:

Ethics workshop (Thursday)

Ethics workshop slides — a workshop discussion of ethical issues in designing ML systems.

Friday lab: Check-ins

Week 12 — Bias and fairness

Nov 17, 19, 20

📌 Due Wednesday, Nov 18: weekly project update — feature importances and crosstabs.

By this week you should be finalizing your modeling results and beginning to look at bias and disparities in your models.

Tuesday: Dealing with bias and fairness in ML

Just as important as assessing whether your model is making accurate predictions is determining whether it is doing so fairly. But what do we mean by fairness? How can you measure it, and what can you do to mitigate disparities you find? Where in your pipeline can bias be introduced? (Spoiler: everywhere.) This week is a brief introduction to the expansive field of algorithmic fairness.

Required (Tuesday):

Optional:

Thursday: Project work

Friday lab: Check-ins

Week 13 — Field trials: validating ML models

Nov 24

📌 Due Monday, Nov 23: weekly project update — bias and disparity results.

No class Thursday or Friday — Thanksgiving break.

This week your group should be continuing to investigate any disparities in your model results, as well as performing any other necessary post-modeling analyses.

Tuesday: Field trials: validating ML models

A model that performs well on historical data has not yet been shown to help an organization achieve its goals. We'll cover how to design a field trial that tests both predictive accuracy in deployment and the effect of acting on those predictions, and what makes such trials hard in policy settings.

Week 14 — Wrap-up and final presentations

Dec 1, 3, 4

Tuesday: Class wrap-up, final check-in, and presentation prep.

Thursday: Final presentations. Each group presents their applied ML project as described on the project page.

Friday lab: Presentation overflow and project work time for teams finishing their reports.

Finals week — final report due

Dec 9

Incorporating the results of your project work throughout the semester as well as feedback from your final presentation, each group writes a final project report.

General background reading

These are useful background but are not required texts: