The abstract concept of “information” can be quantified and this has led to many important advances in the analysis of data in the empirical sciences. This text focuses on a science philosophy based on “multiple working hypotheses” and statistical models to represent them. The fundamental science question relates to the empirical evidence for hypotheses in this set—a formal strength of evidence. Kullback-Leibler information is the information lost when a model is used to approximate full reality. Hirotugu Akaike found a link between K-L information (a cornerstone of information theory) and the maximized log-likelihood (a cornerstone of mathematical statistics). This combination has become the basis for a new paradigm in model based inference. The text advocates formal inference from all the hypotheses/models in the a priori set—multimodel inference.
This compelling approach allows a simple ranking of the science hypothesis and their models. Simple methods are introduced for computing the likelihood of model i, given the data; the probability of model i, given the data; and evidence ratios. These quantities represent a formal strength of evidence and are easy to compute and understand, given the estimated model parameters and associated quantities (e.g., residual sum of squares, maximized log-likelihood, and covariance matrices). Additional forms of multimodel inference include model averaging, unconditional variances, and ways to rank the relative importance of predictor variables.
This textbook is written for people new to the information-theoretic approaches to statistical inference, whether graduate students, post-docs, or professionals in various universities, agencies or institutes. Readers are expected to have a background in general statistical principles, regression analysis, and some exposure to likelihood methods. This is not an elementary text as it assumes reasonable competence in modeling and parameter estimation.
