(Syllabus last updated: 2019-November-21)

Class meetings: Tuesdays, 12.30-3.30pm Demography Seminar Room
Office hours: by appointment (please send me an email and we can find a time)
Email: feehan [at] berkeley.edu
Web: www.dennisfeehan.org/teaching/2019fa_demog260.html
Piazza: https://bit.ly/2lsHm8p


This class will cover topics in the design and implementation of field research for quantitative social science. We will consider studies whose aim is measurement (including surveys), causal inference (including experiments), and prediction (including machine learning). Our goal is to focus on issues of study design and execution, rather than the details of specific tools or methodologies. We will consider examples from a diverse range of different ‘fields’, both digital and physical. Our approach will be to conceptually understand key design and implementation issues for several different types of study design, but it will often be necessary to sacrifice depth for breadth. Thus, our discussion of each design will provide interested students with a useful starting point for deeper study. The class is designed for graduate students who have had at least some exposure to statistics, and who are starting to develop field research projects of their own.

Please re-check the syllabus before you start each week’s reading; it will be updated as the semester progresses

Week Date Theme Topic
1 2019-09-03 Course overview and background Overview; Introduction to sampling
2 2019-09-10 Designing measurement with surveys Fielding a survey in the real world: modes, interviewer training, questionnaire design, cost structure, etc
3 2019-09-17 Complex sampling: stratification, clustering, and unequal probabilities of selection
4 2019-09-24 Complex sampling: multi-stage designs
5 2019-10-01 Sample size estimation; First set of presentations
6 2019-10-08 Survey ethics and the IRB; Field operations; Weighting a survey: non-response, post-stratification, calibration, MRP
7 2019-10-15 (No class)
8 2019-10-22 Guest: Daniel Schneider
9 2019-10-29 Digital data collection Second set of presentations
10 2019-11-05 Linking different data sources Record linkage framework; errors and inference
11 2019-11-12 Record linkage lab
12 2019-11-19 Third set of presentations
13 2019-11-26 Guest; Designing experiments and prediction Power; manipulation checks, pre-registration; prediction and generalization vs parameter estimates
14 2019-12-03 Meet about projects

Requirements and assignments

The requirements of the class are designed to achieve two goals: the first goal is to become familiar with some classic and contemporary issues in the design and implementation of quantitative social science research projects; and the second goal is to write a proposal for a study design. You should think of the proposed design as the first draft of a project that you might be able to continue working on beyond this class.

NB: Please read each week’s articles in the order they are listed on the syllabus

Detailed schedule

Overview and introduction to measurement

Tue, Sep 3

Background for lecture:

  • Robert M. Groves et al., Survey Methodology, vol. 561 (John Wiley & Sons, 2011), Ch. 2.
  • C. E. Sarndal, B. Swensson, and J. Wretman, Model Assisted Survey Sampling (Springer Verlag, 2003), Ch. 2.
  • Sharon L. Lohr, Sampling : Design and Analysis (Chapman and Hall/CRC, 2019), https://www.taylorfrancis.com/books/9780429296284, Ch. 1.

Designing measurement with surveys

Tue, Sep 10 - Fielding a survey

Background for lecture:

  • Groves et al., Survey Methodology, Ch. 8-9.
  • design-based sampling material we discussed last week

Tue, Sep 17 - Complex sampling designs

Background for lecture:

  • Sarndal, Swensson, and Wretman, Model Assisted Survey Sampling, Ch. 3-4.
  • Lohr, Sampling, Ch. 3 and 5.

Topics for lecture:

Tue, Sep 24 - Complex sampling designs, cont

Background for lecture:

  • Sarndal, Swensson, and Wretman, Model Assisted Survey Sampling, Ch. 4.
  • Lohr, Sampling, Ch. 6.

Tue, Oct 1 - Designing a survey and survey paper presentations

We’ll have our first round of paper presentations today.

Background for lecture on basic sample size calculations:

  • Richard Valliant, Jill A. Dever, and Frauke Kreuter, Practical Tools for Designing and Weighting Survey Samples, Statistics for Social and Behavioral Sciences (New York: Springer, 2013), https://tinyurl.com/y4kqgjd3, Ch. 3 and Ch. 9.
  • Lohr, Sampling, Ch. 7.

Tue, Oct 8 - Weighting a survey, non-response, calibration and post-stratification

Ethics and IRB reading for discussion:

Background for lecture on weighting and calibration:

  • Jean-Claude Deville and Carl-Erik Särndal, “Calibration Estimators in Survey Sampling,” Journal of the American Statistical Association 87, no. 418 (June 1992): 376–382, https://www.tandfonline.com/doi/abs/10.1080/01621459.1992.10475217.
  • Valliant, Dever, and Kreuter, Practical Tools for Designing and Weighting Survey Samples, Ch. 6, and Ch. 13-14.
  • Carl-Erik Särndal and Sixten Lundström, Estimation in Surveys with Nonresponse (John Wiley & Sons, 2005).
  • Lohr, Sampling, Ch. 8.

Background for response rates:

Tue, Oct 15 - Class cancelled


Tue, Oct 22 Guest: Daniel Schneider (UCB Sociology)

Prof. Schneider will visit and discuss the design of the SHIFT project, an online survey that he and Prof. Kristen Harknett (UCSF) have been developed and implemented.


Tue, Oct 29 Second round of paper presentations


Possible papers to present on Tue, Oct 29:

NB: In order to claim the specific paper you want to present, please post to the Piazza thread

Anchoring Vignettes

Survey experiments

Wiki surveys

Non-probability sampling

Multilevel Regression and Poststratification

  • Yair Ghitza and Andrew Gelman, “Deep Interactions with MRP: Election Turnout and Voting Patterns Among Small Electoral Subgroups,” American Journal of Political Science 57, no. 3 (2013): 762–776, https://onlinelibrary.wiley.com/doi/abs/10.1111/ajps.12004.
  • Matthew K. Buttice and Benjamin Highton, “How Does Multilevel Regression and Poststratification Perform with Conventional National Surveys?” Political Analysis 21, no. 4 (2013): 449–467, https://www.jstor.org/stable/24572674.

Differential privacy


Linking different data sources

Tue, Nov 5 Lecture on Fellegi-Sunter framework

Background for lecture:

Tue, Nov 12 In-class lab on record linkage

Please be sure to bring your laptops; we’ll be using RStudio for the lab.

Materials TBA


Tue, Nov 19 Third round of paper presentations

Possible papers to present on Tue, Nov 19:

NB: In order to claim the specific paper you want to present, please post to the Piazza thread

Applications of record linkage

Custom online data collection

Using search query data

  • Rediet Abebe et al., “Using Search Queries to Understand Health Information Needs in Africa,” arXiv:1806.05740 [Cs] (June 2018), http://arxiv.org/abs/1806.05740.


Tue, Nov 26 Designing prediction and experiments

Background for lecture:

Useful resources on experimental design from JPAL:


Tue, Dec 3 Guest: Ugur Yildirim / online experiments

Ugur Yildirim will visit and discuss the design and implementation of two online experiments.

Religious Accommodations

Requests to accommodate a student’s religious creed by scheduling tests or examinations at alternative times should be submitted directly to the instructor. Reasonable common sense, judgment and the pursuit of mutual goodwill should result in the positive resolution of scheduling conflicts. The regular campus appeals process applies if a mutually satisfactory arrangement cannot be achieved.

Statement on Academic Freedom

Both students and instructors have rights to academic freedom. Please respect the rights of others to express their points of view in the classroom.

DSP Accommodations

Please see the instructor to discuss accommodations for physical disabilities, medical disabilities and learning disabilities.

Student Resources

The Student Learning Center provides a wide range of resources to promote learning and academic success for students. For information regarding these services, please consult the Student Learning Center Website: https://slc.berkeley.edu/

Academic Integrity

The high academic standard at the University of California, Berkeley, is reflected in each degree that is awarded. As a result, every student is expected to maintain this high standard by ensuring that all academic work reflects unique ideas or properly attributes the ideas to the original sources.

These are some basic expectations of students with regards to academic integrity:

  • Any work submitted should be your own individual thoughts, and should not have been submitted for credit in another course unless you have prior written permission to re-use it in this course from this instructor.
  • All assignments must use “proper attribution,” meaning that you have identified the original source and extent or words or ideas that you reproduce or use in your assignment. This includes drafts and homework assignments!
  • If you are unclear about expectations, ask your instructor or GSI.
  • Do not collaborate or work with other students on assignments or projects unless you have been given permission or instruction to do so.