Bayesian inference [Ling 300]

Klinton Bicknell /// Spring 2016

Bayesian techniques provide a rational way of combining probabilistic information, enabling sophisticated analyses of scientific datasets and principled models of action under uncertainty. This course introduces Bayesian techniques from the ground up.

Schedule Piazza forum

Schedule

Week Date Topic Reading Materials Assignments
1 Mar 31 Introduction, basic probability theory DBDA 1
2 Apr 5, 7 R, more probability theory, Bayes' rule DBDA 4–5 hw1 out Apr 8
3 Apr 12, 14 More R, models for binary outcomes, exact inference DBDA 6 hw1 due Apr 15
4 Apr 19, 21 Markov chain Monte Carlo (MCMC), JAGS DBDA 7–8 hw2 out Apr 22
5 Apr 26, 28 Hierarchical models, more MCMC DBDA 8–10 hw2 due Apr 29
6 May 3, 5 Gaussians, linear models, mixture models DBDA 15–16 hw3 out May 6; Project proposal due May 6
7 May 10, 12 Linear regression with continuous predictors DBDA 17–18 hw3 due May 13 (NOW May 17)
8 May 17, 19 Mixed-effects regression DBDA 19–21 regression code hw4 out May 20
9 May 24, 26 Logistic regression; Bayesian vs. frequentist inference DBDA 11 hw4 due May 27 NOW MAY 30
10 May 31 Computational psycholinguistics
Jun 7 Final project papers due 5pm

Logistics

Course

Time
Tuesdays & Thursdays 9:30–10:50
Location
University Library B182
Textbook
Doing Bayesian Data Analysis, Second Edition (2014) by John Kruschke [DBDA]
Website
kbicknell.github.io/ling300spring2016/

Instructor

Name
Klinton Bicknell
Office hours
Tuesdays 2–3 and by appointment
Office
Linguistics [2016 Sheridan Road] Office 107

Policies

Email
Questions that are not personal should be posted on the Piazza forum (where they can be posted anonymously if desired). To contact the instructor directly, coming to office hours is encouraged. For questions that are personal, students can email the instructor at kbicknell@northwestern.edu.
Description
Hands-on introduction to probability theory, structured probabilistic models, and Bayesian inference, in order to prepare students both to be able to perform sophisticated analyses of data and to construct and evaluate formal computational models instantiating scientific theories, all in the context of the study of language. Students will also learn to use the R language for statistical computing.
Academic integrity
Violations of academic integrity will be referred to the Dean’s office, per WCAS policies. Sanctions can be quite severe, including suspension or permanent expulsion from the university. For details and discussion of how to avoid plagiarism, see the Academic Integrity section of the WCAS undergraduate handbook.

Requirements

Course Grade
  • 70% homeworks (4)
  • 30% final project
Homeworks
There will be four homework assignments throughout the quarter. These assignments will involve a combination of programming exercises and short answer responses. Working together in pairs or small groups when discussing the assignments is encouraged, but each student must code and write up their own homework independently. In addition, students must list on each assignment all students they discussed the assignment with. Homework must be handed in through Canvas.
Final project
Students will complete a final project on a topic related to the course content. There are two possible types of final projects. In one type, the project will involve an in-depth statistical analysis of a dataset using Bayesian techniques. In the other type, the project will involve designing and implementing a probabilistic Bayesian model of a probabilistic inference problem facing some agent (for example, an aspect of language processing or acquisition) and using techniques from the class to determine some of the model's predictions. These projects should be completed in pairs or individually. Students will write short project proposals by the end of week 6, and then a final paper on the project will be due on the first day of finals week.
Keeping up
The syllabus (topics, assignments, due dates) may change. These changes will be announced in class, over email (via Piazza), and on the course website. It is students' responsibility to keep up with them.
Deadlines
All assignments are due at 5pm. For late work turned in between this deadline and 11:59pm the following day (i.e., the first 31 hours after the deadline), I will deduct one percentage point per hour (or partial hour). After the following day, I will give comments and suggestions on work turned in, but you will not receive credit for the assignment. (Of course, if some unusual external circumstance arises which will cause you have trouble meeting a deadline, please contact the instructor as soon as possible.)
AccessibleNU
Any student requesting accommodations related to a disability or other condition is required to register with AccessibleNU (accessiblenu@northwestern.edu; 847-467-5530) and provide professors with an accommodation notification from AccessibleNU, preferably within the first two weeks of class. All information will remain confidential.