Model Selection and Inference: A Practical Information-Theoretic Approach

This book is unique in that it covers the philosophy of model-based data analysis and an omnibus strategy for the analysis of empirical data. The book introduces information theoretic approaches and focuses critical attention on a priori modeling and the selection of a good approximating model that best represents the inference supported by the data. Kullback-Leibler information represents a fundamental quantity in science and is Hirotugu Akaike’s basis for model selection. The maximized log-likelihood function can be bias-corrected to provide an estimate of expected, relative Kullback-Leibler information. This leads to Akaike’s Information Criterion (AIC) and various extensions and these are relatively simple and easy to use in practice, but little taught in statistics classes and far less understood in the applied sciences than should be the case. The information theoretic approaches provide a unified and rigorous theory, an extension of likelihood theory, an important application of information theory, and are objective and practical to employ across a very wide class of empirical problems. Parameter estimation has long been viewed as an optimization problem (e.g., maximize the log-likelihood or minimize the residual sum of squared deviations) and under the information theoretic paradigm, data-based model selection is also an optimization problem. This brings model selection and parameter estimation under a common framework – optimization. The value of AIC is computed for each a priori model to be considered and the model with the minimum AIC is used for statistical inference. However, the paradigm described in this book goes beyond merely the computation and interpretation of AIC to select a parsimonious model for inference from empirical data; it refocuses increased attention on a variety of considerations and modeling prior to the actual analysis of data.