Bayesian Methods:  A Social and Behavioral Approach, Second Edition.

Jeff Gill.
November 2008.
Now available online from Borders and Amazon.com,.

BaM package for R is available here

CRC Press
EMPSR
Recent Review in The American Statistician
Praise for the First Edition:
This book is a brilliant and importantly very accessible introduction to the concept and application of Bayesian approaches to data analysis. The clear strength of the book is in making the concept practical and accessible, without necessarily dumbing it down. The coverage is also remarkable.
-Dr. S.V. Subramanian, Harvard School of Public Health, Cambridge, Massachusetts, USA
One of the signal contributions of Bayesian Methods: A Social and Behavioral Sciences Approach is to reintroduce Bayesian inference and computing to a general social sciences audience. This is an important contribution-one that will make demand for this book high Jeff Gill has gone some way toward reinventing the graduate-level methodology textbook Gill's treatment of the practicalities of convergence is a real service new users of the technique will appreciate this material. the inclusion of material on hierarchical modeling at first seems unconventional; its use in political science, while increasing, has been limited. However, Bayesian inference and MCMC methods are well-suited to these types of problems, and it is exactly these types of treatments that push the discipline in new directions. As noted, a number of monographs have appeared recently to reintroduce Bayesian inference to a new generation of computer-savvy statisticians. However, Gill achieves what these do not: a quality introduction and reference guide to Bayesian inference and MCMC methods that will become a standard in political methodology.
-The Journal of Politics, November 2003
Book Description
The first edition of Bayesian Methods: A Social and Behavioral Sciences Approach helped pave the way for Bayesian approaches to become more prominent in social science methodology. While the focus remains on practical modeling and basic theory as well as on intuitive explanations and derivations without skipping steps, this second edition incorporates the latest methodology and recent changes in software offerings. New to the Second Edition · Two chapters on Markov chain Monte Carlo (MCMC) that cover ergodicity, convergence, mixing, simulated annealing, reversible jump MCMC, and coupling · Expanded coverage of Bayesian linear and hierarchical models · More technical and philosophical details on prior distributions · A dedicated R package (BaM) with data and code for the examples as well as a set of functions for practical purposes such as calculating highest posterior density (HPD) intervals Requiring only a basic working knowledge of linear algebra and calculus, this text is one of the few to offer a graduate-level introduction to Bayesian statistics for social scientists. It first introduces Bayesian statistics and inference, before moving on to assess model quality and fit. Subsequent chapters examine hierarchical models within a Bayesian context and explore MCMC techniques and other numerical methods. Concentrating on practical computing issues, the author includes specific details for Bayesian model building and testing and uses the R and BUGS software for examples and exercises.
ERRATA TO THE FIRST PRINTING (to be fixed with 3rd printing Summer 2009):

  • Page xxi: third line from the bottom, "dedicated the" should be "the dedicated". Thanks to Gary McDonald.
  • Page 1: second paragraph, third line, put "of" before "the". Thanks to Oscar Hernandez.
  • Page 3: second paragraph, third bullet, sentence should start with "Is". Thanks to Oscar Hernandez.
  • Page 4: third line of section 1.2, remove "the" before "null". Thanks to Oscar Hernandez.
  • Page 6: section 1.3, line 8, change "[2006]" to "(2006)" and remove ")" at the end of the sentence. Thanks to Oscar Hernandez.
  • Page 7: seventh line from the bottom, change the last word from "even" to "event". Thanks to Mark Fisher.
  • Page 9: equation (1.1), change "E[X]" to "E[h(X)]". Thanks to Mark Fisher.
  • Page 36: the 8th line of R code should be: y <- c(y,cand.val[2]) and the 12th line should be: y <- c(y,y[i])
  • Page 38: Problem 1.8, change "Example 1.5.1" to "Section 1.5.1".
  • Page 42: second paragraph, line 6, the square is missing in the variance formula. Thanks to Oscar Hernandexz.
  • Page 44: first line after equation (2.6), change p(\theta)L(\theta|D)d\theta to p(\theta)L(\theta|X)d\theta. Thanks to Jamie Gertsch.
  • Page 46: Example 2.1, line 4, change "Example 1.5.1" to "Exercise 1.8". Thanks to Hong Min Park.
  • Page 46: equation (2.8), change "p(" to "\pi(". Thanks to Gary McDonald.
  • Page 50: equation (2.12), the third and fourth posteriors (pi) are CDFs.
  • Page 57: equations (2.18) and (2.19), all L() functions should have theta before x. Thanks to Hong Min Park.
  • Page 66: Exercise 2.1, second to last line of the page, insert "a" between "specify" and "normal".
  • Page 69: Exercise 2.8, change "confidence set" to "credible set". Thanks to Gary McDonald.
  • Page 79: equations after "Begin with" change \mu to \mu_{\sigma^2} and \sigma^2 to s^2_{\sigma^2}, same for         sentence beginning "Setting both". Thanks to Hong Min Park.
  • Page 79: line 4, change "both \mu and \sigma^2" to "both \mu and \sigma". Thanks to Gary McDonald.
  • Page 79: second line of equation (3.13), swap the order of sigma^2/s_0 and m in the p() statement.
  • Page 81: first equation add conditionality on \sigma^2 to LHS, and add conditionality on x to RHS of equation              (3.19). Thanks to Hong Min Park.
  • Page 82: equation (3.20), line 3: 2\x_i should be 2\x_i', line 5: 2n\bar{x} should be 2n\bar{x}'. Thanks to Hong Min        Park. Line after long equation starting with "Since", move the ")" from after \bar{x'} to after \x' (same in last expression of formula above). Thanks to Gary McDonald. Same line starting with "Since", change "S^2)" to "S^2".
  • Page 83: LHS of posterior statement (3.23), condition on x. Second line of same equation make first (\mu-m) a             transpose (\mu-m)'. Thanks to Hong Min Park.
  • Page 92: third line of equation (3.29), s^2 should be s^n. Thanks to Hong Min Park.
  • Page 93: first line, change "works out very nicely:" to "works out very nice, provided sufficient sample size:". First sentence after equation (3.32), the first parameter of IG should be (n-1)/2 not (n-2)/2. Thanks to Gary McDonald.
  • Page 95: fourth line of R code: add [,1] after to iterations of iq. Thanks to Hong Min Park.
  • Page 96: equation (3.35), change x_j to x_i. Thanks to Hong Min Park.
  • Page 97: equation (3.37), second line, change \mu to \mu_j. Thanks to Hong Min Park.
  • Page 101: n.post1 function: change "credible.int<-c(mu.hat-1.96*sigma.hat,mu.hat+1.96*sigma.hat)" to "credible.int<-c(mu.hat-1.96*sqrt(sigma.hat),mu.hat+1.96*sqrt(sigma.hat))". Thanks to Michael Scharkow
  • Page 113: first equation, second line, strike the equal sign. Thanks to Hong Min Park.
  • Page 113: first line after equation (4.17), change "n+a-k" to "n+a-2-k". Thanks to Hong Min Park. Last line, change \sigma to \sigma^2.
  • Page 114: first line of Table 4.3: change "t(n+a-k)" to "t(n+a-k-2)".
  • Page 117: equations (4.20) and (4.21), the "\pi()" expressions with \tilde(y) should also have \tilde(X). Thanks to Gary McDonald.
  • Page 123: equation (4.32), first line, strike the equal sign. Thanks to Hong Min Park.
  • Page 129: exercise 4.7, last line before table, change "priors" to "posteriors". Thanks to Hong Min Park.
  • Page 131: second to last line of code, change "-2" to "-3". Thanks to Hong Min Park. (Same for identical code on page 226.)
  • Page 132: before first line of code, add alpha <- nrow(x)+A-ncol(X); beta <- 0.5*S2*(nrow(X)+A-ncol(X)). Thanks to Hong Min Park. Line 2 of the code, change "-3" to "-2". Thanks to Gary McDonald. (Same for identical code on page 226.)
  • Page 145: last line (equation), change (1) to c, and the last equality to \propto. Thanks to Oscar Hernandez.
  • Page 146: equation (5.7), integral limits of 1/2 should be 1. Thanks to Gary McDonald.
  • Page 148: equations (5.9) and (5.10), change the equalities to proportions.
  • Page 148: equation (5.10), there are three \theta's that should be bold and the derivative symbols should indicate partial derivatives (curly d's). Thanks to Oscar Hernandez.
  • Page 149: first equation, change n choose p to n choose x. Thanks to Hong Min Park.
  • Page 149: first equation, change 0 /ge p /ge 1 to 0 \le p \le 1. Thanks to Oscar Hernandez.
  • Page 150: first equation, the last x_i should be x_i!. Thanks to Hong Min Park.
  • Page 150: third and fourth equations, change "d^2\lambda" to "d\lambda^2". Thanks to Oscar Hernandez.
  • Page 152: table near the bottom of the page, line starting v(\beta), change the last part from \theta_i^2 to d\theta_i^2.     Thanks to Hong Min Park.
  • Page 153: line 5, change "down to $\pi(\B|\X,\y,\phi) = |\X'\X|^{\half} \propto c$" to "down to $p_{J}(\B) \propto |\X'\X|^{\half} = c$". Thanks to Gary McDonald.
  • Page 153: in Section Reference Priors, sixth line, change Taio to Tiao. Thanks to Oscar Hernandez.
  • Page 154: line 4, change "prior" to "likelihood". Thanks to Saiwing Yeung.
  • Page 159: line 5 of the first paragraph, change "subjective" to "subject". Thanks to Gary McDonald.
  • Page 164: the last three lines should read: "$\alpha=1506,\, \beta=953$. This means that the normal prior mean is obtained by $x_{0.50} = 1506 + 0.953(0) = 1506$, and the normal prior standard deviation is obtained by $x_{0.84} = 0.953\Phi(0.84) = 762$."
  • Page 172: second to last line, change "Y-b" to "Y-a". Last line, change "+b" to "+a". Thanks to Gary McDonald.
  • Page 180: second paragraph, change "\alpha-i \ge 0" to "\alpha_i > 0". Thanks to Gary McDonald.
  • Page 180: equations (5.38), (5.39), (5.40), change subscript on all summations to be "k=1". Thanks to Gary McDonald.
  • Page 186: exercise 5.6, the reference should be to exercise 5 not exercise 4. Thanks to Hong Min Park.
  • Page 186: exercise 5.7, third to the last line, change "\rho^2" to "\rho^s". Thanks to Hong Min Park.
  • Page 191: fourth line from the bottom, insert "been" after "already". Thanks to Oscar Hernandez.
  • Page 193: second to last sentence above new section, insert "(see page 265)" after "distance". Thanks to Gary McDonald.
  • Page 200: equation (6.3), both \sigma's should be \sigma_0. Thanks to Hong Min Park.
  • Page 200: equation (6.3), remove "=1". Thanks to Oscar Hernandez.
  • Page 200: line 5: change 10,000 to 1,000. Thanks to Oscar Hernandez.
  • Page 202: last line of 2nd paragraph, change "prior" to "posterior". Thanks to Gary McDonald.
  • Page 203: equation (6.4), put "N" before the open bracket. Thanks to Gary McDonald.
  • Page 204: second paragraph, second line: change beta to gamma. Also on Page 205: same for caption of Figure 6.3. Thanks to Oscar Hernandez.
  • Page 204: third paragraph, fifth line, change to \alpha/\beta^2. Thanks to Hong Min Park.
  • Page 218: text before (6.21), strike the first "is". Thanks to Gary McDonald.
  • Page 219: line 14, \theta_i should be \theta_k. Thanks to Gary McDonald.
  • Page 220: equations (6.27) and (6.28), hats not necessary on \theta.
  • Page 223: Exercise 6.4, the reference should be to exercise 4.4 not 4. Thanks to Hong Min Park.
  • Page 224: Exercise 6.6, last line of second paragraph, change both \sigma^2 to \sigma^{-2}. Thanks to Hong Min Park.
  • Page 234: first full sentence, change "simple two-sided" to "simple one-sided" and change reference from (7.2) to (7.1). Thanks to Hong Min Park.
  • Page 236: four lines after equation 7.5, put a minus sign before the third 1/2 (exponent of (1-p)). Thanks to Oscar Hernandez. Third line from the bottom in equation (7.6), change "r+y_i+1" to "r+y_i-1". Thanks to Gary McDonald.
  • Page 238: last line, remove phrase "in a general manner". Thanks to Oscar Hernandez.
  • Page 244: all of the betas on this page should be bold (fix 3).
  • Page 247: first line, insert "for normal data" after "t-test". Thanks to Gary McDonald.
  • Page 266: in equation (7.37) change the integrals to sums. Thanks to Oscar Hernandez.
  • Page 268: equation (7.43), change x_0 to \theta_0, change f to h, change \theta_1 to \theta, change R to R_n. Thanks to Oscar Hernandez.
  • Page 268: equation (7.44), change f to h. Thanks to Oscar Hernandez.
  • Page 268: second line after equation (7.43), change R to R_n. Thanks to Oscar Hernandez.
  • Page 269: equation (7.45), last two lines, sigma's need hats. Thanks to Oscar Hernandez.
  • Page 271: put hats on \theta_1 and \theta_2 in the paragraph before exercises. Thanks to Gary McDonald.
  • Page 276: first sentence of first full paragraph, change "Metropolis )" to "Metropolis)". Thanks to Mark Fisher.
  • Page 277: second to last line, change "0.075" to "0.975". Thanks to Mark Fisher.
  • Page 278: equation (8.6), E[h(\theta)|x)] should be E[h(\theta|x)]. Thanks to Jonghhoon Eun.
  • Page 279: third equation on the page, change x_i to log(x_i). Thanks to Hong Min Park.
  • Page 286: two lines up from Section 8.3.2, change 12.56952 to 0.4816.
  • Page 290: last line, remove the k before g. Thanks to Oscar Hernandez.
  • Page 305: 9 lines from the bottom: insert "the" after "that". Thanks to Oscar Hernandez.
  • Page 306: equation (8.33), after the summation sign, change x_i to log(x_i). Thanks to Oscar Hernandez.
  • Page 308: line 17, change w_{[j}] to w_{[j]}. Thanks to Gary McDonald.
  • Page 309: third line, change "1,100s" to "1,000s". Thanks to Gary McDonald.
  • Page 314: E-Step, take out "log". Thanks to Hong Min Park.
  • Page 318: line before last equation, change "(8.50)" to "(8.52)". Thanks to Colin Hubbard.
  • Page 322: line before equation, change "(k+1)" to "(k)". Thanks to Hong Min Park.
  • Page 337: Exercise 8.10, change "3+" to "3-10".
  • Page 343: last word of first paragraph, change "afterwords" to "afterwards". Thanks to Oscar Hernandez.
  • Page 344: first sentence of last paragraph, change "only the" to "only on the". Thanks to Gary McDonald.
  • Page 345: fifth line from the bottom, strike the second "given". Thanks to Gary McDonald.
  • Page 347: ten lines from the bottom, change "port" to "part".
  • Page 359: first paragraph, 0.723 is the posterior mean value. Thanks to Gary McDonald.
  • Page 361: middle paragraph, change "product of its parameters" to "ratio of its parameters" and "(\alpha\beta)" to "(\alpha/\beta)", "(\delta\gamma)" to "(\delta/\gamma)". Thanks to Hong Min Park.
  • Page 362: third line of equation (9.14), in first fraction change "y_i" to "y_i!" in the denominator. Also, fourth line after equation (9.15), remove "p(k)". Thanks to Hong Min Park.
  • Page 364, line five, change "proceed" to "proceeds". Thanks to Gary McDonald.
  • Page 369: seventh line from the bottom, insert "to" after "simplifies." Thanks to Gary McDonald.
  • Page 371: fourth line from the bottom, take the extra comma out of of q(). Thanks to Will Beasley.
  • Page 372: second line from the bottom, change "less than r" to "less than \alpha(\theta',\theta)". Thanks to Gary McDonald.
  • Page 373: equation (9.26), remove comma from denominator. Thanks to Will Beasley.
  • Page 373: three lines below equation (9.26), change the comma in the second q() function to a "|". Thanks to Will Beasley.
  • Page 374: equation (9.28), add a closing parenthesis to the end. Thanks to Will Beasley.
  • Page 374: last paragraph, change "Cauchy" to "Student's-t". Thanks to Hong Min Park.
  • Page 398: fourth line, in multinomial expression "p_it" should be "p_{it}", fifth line after equation (10.4), put a ")" before "=", second to last line of first paragraph, change "106" to "107".
  • Page 398: Model 1 and Model 2, there is a ")" missing. Thanks to Hong Min Park.
  • Page 399: Model 3 to Model 6, there is a ")" missing. Thanks to Hong Min Park.
  • Page 400: four lines after equation (10.7), change "is" to "has". Thanks to Oscar Hernandez.
  • Page 402: equation (10.11), change "f,g" to "g_1,g_2", and in following paragraph change "I_F(f(\theta_i),g(\theta_{i+k}))" to "I_F(g_1(\theta_i),g_2(\theta_{i+k}))". Thanks to Hong Min Park.
  • Page 402: seventh line from the bottom, remove ")". Thanks to Oscar Hernandez.
  • Page 403: second line after equation (10.12), make the G \mathcal font. Thanks to Gary McDonald.
  • Page 403: last line of equation (10.12), change gamma to tau. Thanks to Oscar Hernandez.
  • Page 404: line 4, change "following" to "supplied". Thanks to Oscar Hernandez.
  • Page 415: last line of equation (10.18), \eta should be subscripted by k and in the line below, "kth" should be "jth". Thanks to Hong Min Park.
  • Page 418: equation (10.21), BE should be BR. Thanks to Hong Min Park.
  • Page 430: exercise 10.7, first line of equations change "\theta_j" to "\beta_i,\lambda_i", and add at the end of the exercise: "with \X generated in \R from MVN((1,2,3), \Sigma), \Sigma = 2I along with a leading column of 1s". Thanks to Hong Min Park.
  • Page 433: second line of Section 11.2, change "P" to "\mathfrak{P}" in the probability space. Thanks to Hong Min Park.
  • Page 434: first line, insert "non-overlapping" before "multiple".
  • Page 436: second line of Section 11.3.3, change "value of n" to "value of t". Thanks to Hong Min Park.
  • Page 464: last sentence of first full paragraph, insert a space before "This". Thanks to Mark Fisher.
  • Page 468: sixth line from the bottom, change "Markov chains" to "Markov chain". Thanks to Gary McDonald.
  • Page 470: first sentence of first full paragraph, insert "the square root of" between "given by:" and "sample                   variance". Thanks to Mark Fisher.
  • Page 476: first sentence after equation (12.10). Fix definitions to: \bar{g}(\theta_1)=\sum_{i=1}^{n_1}g(\theta_i)/n_1$, $\bar{g}(\theta_2)=\sum_{i=1}^{n_2}g(\theta_i)/n_2. Thanks to Hong Min Park.
  • Page 484, first sentence in Example 12.7. Fix to: q=0.025. Thanks to Hong Ming Park.
  • Page 501: second to last sentence before equation (12.29), change "with much" to "with not much". Thanks to Gary McDonald.
  • Page 501: equation (12.29). In the denominator, change \theta_m to \theta_N. Thanks to Hong Min Park.
  • Page 502: equation (12.31), first expected value needs an extra ")". Thanks to Gary McDonald.
  • Page 503: second to last paragraph, the summation should be: \frac{1}{k-1}\sum_{j=1}^{k-1}. Thanks to Hong Min Park.
  • Page 504: first line, summation should be: \frac{1}{k-1}\sum_{j=1}^{k-1}. Thanks to Hong Min Park.
  • Page 513: in the algorithm steps, (1) "At the t^{th} step draw" not "At the j^{th} step draw", (2) change T_k to T_t in two places and T_{k+1} to T_{t+1} in one place. Thanks to Hong Min Park.
  • Page 520: Step N+1, the second theta' should have the subscript "N" not "N+1". Thanks to Hong Min Park.
  • Page 531: paragraph 2, line 5, change "and algorithm" to "an algorithm". Thanks to Gary McDonald.
  • Page 538: in line seven, convergence is in distribution, so D in place of P above the arrow. Thanks to Oscar Hernandez.
  • Page 538: footnote, third line from the bottom, the square missing in the formula for the variance. Thanks to Oscar Hernandez.
  • Page 539: last line, change -2y'Xb to -2bX'y.
  • Page 540: first equation, change -2y'X to -2X'y.
  • Page 544: first line of equation A.11, remove summation. Thanks to Oscar Hernandez.
  • Page 568: Dirichlet distribution, change 0 \ge x_i \ge 1 to 0 \le x_i \le 1. Thanks to Oscar Hernandez.
  • Page 570: t, Multivariate distribution, change: Sigma to M, "half" to 1/2, and -k to -k/2. Remove the x after the last M. Thanks to Hong Min Park.