Probability & Statistics Qualifying Exams
The following topics & references will prepare you for the exam.
Topics for Mathematical Statistics
- Classical parametric families of distributions. Moment generating functions. The distribution of functions of random variable, the Jacobian method.
- The Method of Moments: basic properties and asymptotics.
- Exponential families: canonical and curved. Canonical exponential families:; natural parameter space, identifiability, convexity. Location and scale families.
- Sufficient, minimal, ancillary, complete statistics. Factorization theorem, conditions for minimality or completeness.Sufficiency theory for canonical exponential families. Basu's theorem.
- Maximum Likelihood estimation: invariance of the ML estimator. General conditions for existence and uniqueness of the ML estimator. ML estimation for canonical exponential families.
- UMVU estimation: risk and loss functions, Information Inequalities. Fisher information, uniqueness of the UMVU estimator. Rao-Blackwellization of estimators. Risk-function-based optimality and sufficiency. Information inequalities for exponential families.
- Hypothesis Testing: simple and composite hypotheses, randomized and non-randomized tests, size, power, p-values. The Neyman-Pearson Lemma. Optimality of tests: UMP and UMPU tests. Generalized Likelihood Ratio tests. Asymptotics of the likelihood ratio tests.
- Asymptotics: Convergence in law and in probability, the Delta Method and Taylor approximations. Consistent roots for the likelihood equation. Asymptotic efficiency, asymptotics of the ML estimator for canonical exponential families.
1. Casella and Berger (2002), Statistical Inference, 2nd edition, Duxbury
2. Lehmann, E., and G. Casella (1998), Theory of Point Estimation, John Wiley and Sons, Inc., 2nd edition
Topics for Stochastic Processes
- Generating functions, Abel's theorem, linear difference equations
- Random walks: conditional distributions, gambler's ruin problem, hitting time probabilities, Reflection Principle, sojourn times and arcsine laws.
- Discrete-time Markov Chains: conditional distributions, transition matrices, Chapman-Kolmogorov equations, mean recurrence time. The classification of states (persistent, transient, positive, null; irreducibility, periodicity). Stopping times, the Strong Markov property. Stationary distributions and asymptotics. Reversibility. The Perron-Frobenius theorem.
- Birth processes and the Poisson process, the infintesimal approach.
- Continuous-time Markov Chains: conditional distributions, stochastic semigroups, rate matrices (generators), Kolmogorov's forward and backward equations. Levy's Dichomoty. The Ergodic theorem. Jumps chains, classification of states (instantaneous, stable, absorbing), distribution of holding times.
- Introduction to diffusions within the infinitesimal approach, the Ornstein-Uhlenbeck process, Brownian motion with drift. Gaussian processes. Brownian motion: basic properties, relation to random walks, distributional properties.
1. Grimmett, G. R., and D. R. Stirzaker (2001), Probability and Random Processes, Oxford Science Publications, 3rd edition.
2. Resnick, S. (1992), Adventures in Stochastic Processes, Birkhauser Boston
Topics for Linear Models
- Multiple and Simple Least Squares Linear Regression developed from Linear Algebra, with emphasis on the geometric interpretation and Perpendicular Projection Matrices, orthogonal matrices, t and F statistics from this viewpoint.
- Gauss Markov Theorm
- Sampling Distributions and Statistics Inference for Regression Parameters
- F tests for nested models and the general linear hypothesis
- Weighted Linear Regression
- One-way and Two-way ANOVA
- Identifiability and Estimability of Parameters
- Regression Diagnostics--residuals, plot and heteroscedasticity, leverage, outliers, influential points, Cook's distance
- Model Section and Multicollinearity--R squared, Mallows Cp, AIC, BIC, forward and backward and stepwise selection
1. Christensen, Plane answers to Complex Questions
2. Weisberg, Applied Linear Regression