An Introduction to Bayesian Inference and Decision by Robert L. Winkler

Currently in its 2nd Edition, 2nd Printing



Bayesian Inference and Decision

The basic concepts of Bayesian inference and decision have not really changed since the first edition of this book was published in 1972. Even so, Bayesian inference and decision has been a very fertile and rapidly growing field, both in terms of theoretical/methodological research and in terms of real-world applications.

This book gives a foundation in the concepts, enables readers to understand the results of analyses in Bayesian inference and decision, provides tools to model real-world problems and carry out basic analyses, and prepares readers for further explorations in Bayesian inference and decision. In the second edition, material has been added on some topics, examples and exercises have been updated, and perspectives have been added to each chapter and the end of the book to indicate how the field has changed and to give some new references.



Publication Information: List Price $59.95

Shipping - select Standard (USPS Media), Priority (USPS Priority Mail), Ground (FedEx Ground), or Overnight (unlike prime, we have to charge you for shipping). We have to collect Sales Tax for FL and TX sales.

Canada Orders:

Priority Mail is $24.95. The most cost and time effective shipping method is eBay; we will set up an eBay sale for you if you want to proceed this way.

International Orders:

Please contact us for international shipping. The most cost and time effective shipping method is eBay; we will set up an eBay sale for you if you want to proceed this way.

Purchase via PayPal

No PayPal account required - PayPal will accept your credit card. Please specify a shipping method when you check out.

About this book...

ISBN Number is 0964793849. Published January, 2003. Second printing: August, 2010.

Pages: 464 plus supplemental material available: click here to download (23 MB Zip File).

Academic Usage

Many colleges and universities are using An Introduction to Bayesian Inference and Decision as a graduate level textbook for Bayesian Analysis and/or advanced topics in Decision Analysis. The files contained within the supplemental material contain PowerPoint files with all of the graphics from the textbook, which serve as a starting point for lecture development.

Reviews

About An Introduction to Bayesian Inference and Decision

The first printing of this book was done using two interior colors. It came out beautifully; however, with the second printing in 2010, we had one decision to make: either raise the price significantly (original list price: $49.95) or change to monochromatic text (the first printing’s text utilized two colors). We decided that keeping the price at $59.95 was probably more important than two colors, so the text of this printing is in black and grayscale. There were a few minor errors that we also corrected. We were very satisfied with the quality of the grayscale.

However, we did keep a few of the two-interior-color copies - if you would like to buy one of these, please let us know!

Concerning the first printing, 2002...

During one of the sessions at the Decision Analysis Affinity Group (DAAG - www.daag.net) conference in Orlando (January, 1999), the question was asked, “What is a good reference on Bayesian statistics?” The session panel members all agreed on the answer: “Bob Winkler’s book - but it is out of print.” Bob Clemen of Duke University (author of Making Hard Decisions) and I were talking about this during the next break, and Bob mentioned that he knew Bob Winkler (also at Duke) and that Bob might be interested in publishing a second edition. As our vision at Probabilistic Publishing is to keep key decision and risk analysis publications available for people interested in the field, this piqued my interest quickly. We eventually were able to borrow a copy of the first edition via the Florida library system. We were very impressed with the clarity of Bob’s writing and agreed that this book should be kept in publication. Thanks to Bob Clemen’s help, Professor Winkler and I traded a few phone calls and e-mails and finally met for breakfast here in Gainesville and decided to proceed with the project. Then the fun began!

Based on a discussion with Sandee Cohen (author of several excellent books on desktop publishing) at MacWorld Expo, we decided to use Adobe® Framemaker for the text (we had used Adobe® Pagemaker for our previous publications). This has proved to be a bit of an adventure. We are thankful to Bob for his patience, as it took us much more time to set the equations and complete the graphics than we thought it would. However, the more we worked with Bob’s material, the more our vision of keeping this work available was confirmed.

We’re very appreciative of the many hours Aliza Bar-David spent proofing text and entering numbers and for the consistent high quality of her work. I’d also like to thank David Skinner and Gary Bush of Decision Strategies for their encouragement with the project. And thank you Tom Sciance for getting us interested in decision analysis in the first place!

Concerning the second printing, 2010...

We’re pleased that the first printing sold out. We’re especially appreciative of those professors who use this textbook even though we can’t provide the perks that the major publishing houses provide. Hopefully the reasonable price at which we can offer this text to your students will partly compensate for this.

Another humorous thing happened while we were preparing for the second printing. I asked Heather Reitz, a professional graphic artist who did the cover design for Game Theory for Business for us if she had any ideas how we could improve the cover relative to the first printing. I sent her a copy of the book. Her initial comment was, "If I saw a book with a cover like yours at a book store, I wouldn't buy it no matter how good a book it was." This honest appraisal of my graphic design capability (rather the lack thereof) was very sobering. We're hoping the design for the second printing is an improvement! Just for grins, I'm posting a (very small) picture of the first printing cover to the right so you can see what it looked like. I didn't think it was that bad, but I guess that is why I'm an engineer and not a graphic artist!

Bayesian Inference and Decision

From the Book: Preface

From the Preface to the First Edition

This book is intended as an introduction to statistical inference and decision from a Bayesian viewpoint. The emphasis is largely conceptual; although specific classes of situations are considered, the primary objective is to present a general framework for handling problems of statistical inference and decision and to develop an appreciation for the basic concepts and the theory underlying this framework. Furthermore, although a considerable amount of space is devoted to the solution of problems once they have been expressed in terms of the general framework presented here, the expression of real-world situations in terms of the desired framework (that is, the modeling precess) and the determination of the necessary inputs are also discussed at some length.

The mathematical prerequisite is college algebra; calculus is used in a few places, but the reader unfamiliar with calculus can easily skip over the technical details without loss of continuity. Furthermore, no previous knowledge of probability and statistics is assumed. However, those who have been exposed to more “traditional” statistics courses may find this book of interest, and their previous training may enable them to proceed at a faster rate, particularly in the early chapters, than those with no exposure to probability and statistics. In general, considerable flexibility in the choice and emphasis of topics is possible, and an instructor can take advantage of this flexibility to “fit” the book to a particular course and to a particular group of students. For example, some courses might be primarily inference oriented, whereas others might be primarily decision oriented; with regard to different groups of students, there may be variability in general interests as well as in mathematical and statistical backgrounds.

The book is introductory in that it requires no previous exposure to statistics, and it is self contained in that it can be read without referring to other sources. However, references are given at the end of each chapter for readers interested in further discussions of any topic (at an elementary, intermediate, or advanced level), in more detailed and more concrete applications, in more complete theoretical developments, in extensions to models not considered here, in discussions of related topics, in documentation of historical developments, and so on. These references increase the flexibility of the book, for a judicious choice of readings to accompany the book can vary the emphasis and level considerably. As noted above, however, the book is self-contained and can be read without consulting any of the references.

Numerous problems are included at the end of each chapter, ranging from straightforward applications of the textual material to problems that require considerably more thought. The problems are an integral part of the book, serving to reinforce the reader’s grasp of the concepts presented in the text and to point out possible applications and extensions of these concepts. Answers to selected problems are presented at the end of the book.

Changes in the Second Edition

The basic concepts of Bayesian inference and decision covered in this book have not really changed since the first edition of this book was published. As a result, the changes from the First Edition are quite minor, and the preceding comments from the Preface to that edition still apply to the Second Edition. Material has been added on a few topics, some examples and exercises have been updated, a “Perspectives” section has been added at the end of each chapter to indicate how the field has changed and to give some new references, and some brief “Concluding Perspectives” are given at the end of the book.

This is not to say that there have not been new developments in the field in the past thirty years. To the contrary, Bayesian inference and decision has been a very fertile field, both in terms of theoretical/methodological research and in terms of real-world applications. To do justice to all of these new developments in a coherent fashion at the level of this book would be a difficult and lengthy project that would expand the scope, level, and length of the book far beyond the original intent of providing an introduction to the field at a level accessible to a reasonably wide audience. Rather than undertake such a project, I continue to think of this book as an introduction to the basic underlying concepts of Bayesian inference and decision. As such, it attempts to do the following:

     * give a solid foundation in these basic concepts,
     * enable readers to understand the results of analyses involving Bayesian inference and decision,
     * provide some tools to allow readers to model simple real-world problems and to carry out basic analyses,
     * prepare readers for further explorations in Bayesian inference and decision.
     

With respect to further explorations, the literature in Bayesian inference and decision and related fields has exploded in the last three decades and is spread across a wide variety of outlets from many different disciplines. The creation of anything even slightly resembling an exhaustive list of references is a daunting task. Therefore, unlike the extensive list of references in the First Edition, most of which are still included in the current edition and many of which are still very relevant and valuable, the new references are fewer in number. They are designed to reflect both the breadth and depth of developments in the field and to provide some key entry points to the recent literature for those who are interested.

A new feature of the Second Edition is a CD. The CD includes tables that were in the back of the book in the First Edition. In some cases, these are extended via easy-to-use routines to calculate probabilities for even wider choices of parameters than available in the tables. Also on the CD are key tables and figures from the book and some work-throughs of examples in the book. The Exercises from Chapters 2 through 7 are also included in Microsoft Word format on the CD to facilitate handing in homework electronically. [Note: CD materials are now available to anyone via download from this site. Use link at the top of this page.]

With the usual disclaimer, I would like to thank the many users of the book (faculty, students, and practitioners) who offered useful suggestions regarding the book over the years. I particularly appreciate the many faculty who asked for permission to copy the book and students who had to struggle with those copies instead of bound books after the First Edition went out of print! Special thanks are due to Bob Clemen of Duke University and Dave and Debbie Charlesworth of Probabilistic Publishing. Bob suggested to Dave that he might consider publishing a new edition of this book, and Dave approached me with the idea. It has been a pleasure to work with Dave and Debbie in the development and production of the Second Edition. Much of my work on this edition was done at Duke, but I am also grateful for the hospitality of INSEAD-Singapore and INSEAD-Fontainebleau, where I put finishing touches on the book while on sabbatical from Duke.

Robert L. Winkler, October 2002, Paris, France


From the Book: Table of Contents


1 Introduction 1
      Perspectives 3
2 Probability: Measuring Uncertainty 5
  2.1 The Mathematical Theory of Probability 5
  2.2 The Frequency Interpretation of Probability 10
  2.3 The Subjective Interpretation of Probability 14
  2.4 Probabilities, Lotteries, and Betting Odds 17
  2.5 Probability and Decision Making 23
  2.6 The Conditional Nature of Probability 27
  2.7 Bayes’ Theorem 36
  2.8 References and Suggestions for Further Reading 42
      Perspectives 42
      Exercises 43
3 Bayesian Inference for Discrete Probability Models 49
  3.1 Discrete Probability Models 50
  3.2 Bayes’ Theorem for Discrete Probability Models 64
  3.3 Discrete Prior Distributions: A Bernoulli Example 70
  3.4 Discrete Prior Distributions: A Poisson Example 78
  3.5 The Assessment of Prior Probabilities 83
  3.6 The Assessment of Likelihoods 90
  3.7 The Predictive Distribution 99
  3.8 References and Suggestions for Further Reading 104
      Perspectives 105
      Exercises 106
4 Bayesian Inference for Continuous Probability Models 117
  4.1 Continuous Probability Models 117
  4.2 Bayes’ Theorem for Continuous Probability Models 127
  4.3 Conjugate Prior Distributions for the Bernoulli Process 132
  4.4 The Use of Beta Prior Distributions: An Example 140
  4.5 Conjugate Prior Distributions for the Normal Process 143
  4.6 The Use of Normal Prior Distributions: An Example 153
  4.7 Conjugate Prior Distributions for Other Processes 158
  4.8 Assessment of Prior Distributions 161
  4.9 Discrete Approximations of Continuous Probability Models 170
  4.10 Representing a Diffuse Prior State 175
  4.11 Predictive Distributions 180
  4.12 The Posterior Distribution and Decision Theory 184
  4.13 References and Suggestions for Further Reading 185
       Perspectives 186
       Exercises 188
5 Decision Theory 197
  5.1 Certainty Versus Uncertainty 198
  5.2 Payoffs and Losses 200
  5.3 Nonprobabilistic Criteria for Decision Making Under Uncertainty 208
  5.4 Probabilistic Criteria for Decision Making Under Uncertainty 212
  5.5 Utility 217
  5.6 The Assessment of Utility Functions 220
  5.7 Utility and Subjective Probability 233
  5.8 Utility and Decision Making 236
  5.9 A Formal Statement of the Decision Problem 239
  5.10 Decision Making Under Uncertainty: An Example 242
  5.11 References and Suggestions for Further Reading 253
       Perspectives 255
       Exercises 256
6 The Value of Information 267
  6.1 Terminal Decisions and Preposterior Decisions 267
  6.2 The Value of Perfect Information 268
  6.3 The Value of Sample Information 276
  6.4 Preposterior Analysis with Nonlinear Utility 292
  6.5 Sequential Analysis 301
  6.6 Linear Payoff Functions: The Two-Action Problem 316
  6.7 Loss Functions and Linear Payoff Functions 321
  6.8 Linear Loss Functions and the Normal Distribution 327
  6.9 Linear Loss Functions and the Beta Distribution 332
  6.10 The General Finite-Action Problem 335
  6.11 References and Suggestions for Further Reading 340
       Perspectives 341
       Exercises 342
7 Inference and Decision 351
  7.1 Diffuse Prior Distributions and Classical Statistics 353
  7.2 The Posterior Distribution and Estimation 356
  7.3 Decision Theory and Point Estimation 361
  7.4 Point Estimation: Linear and Quadratic Loss Functions 364
  7.5 Prior and Posterior Odds Ratios 373
  7.6 The Likelihood Ratio and Classical Hypothesis Testing 378
  7.7 The Posterior Distribution and Hypothesis Testing 382
  7.8 Decision Theory and Hypothesis Testing 391
  7.9 The Different Approaches to Statistical Problems 395
  7.10 References and Suggestions for Further Reading 396
       Perspectives 397
       Exercises 398
Concluding Perspectives 407
Tables 411
       Table 1 Poisson Probabilities 411
       Table 2 Cumulative Standard Normal Probabilities 417
       Table 3 Standard Normal Density Function 420
       Table 4 Unit Normal Linear Loss Integral 422
       Table 5 Random Digits 424
Bibliography 425
Answers to Selected Exercises 442
Index 447