A detailed look at how to fit a robust GLS model on heteroskedastic, auto-correlated data sets — Generalized Least Squares (GLS) estimation is a generalization of the Ordinary Least Squares (OLS) estimation technique. GLS is especially suitable for fitting linear models on data sets that exhibit heteroskedasticity (i.e., non-constant variance) and/or auto-correlation. Real world data sets often exhibit these characteristics making GLS a very useful alternative to…

How to use the White’s heteroskedasticity consistent estimator using Python and statsmodels — In this article, we shall learn how to employ the HC estimator to perform statistical inference that is robust to heteroskedasticity. This article is PART 2 of the following two part series: PART 1: Introducing White’s Heteroskedasticity Consistent Estimator PART 2: A tutorial on White’s Heteroskedasticity Consistent Estimator using Python and…

An introduction to the HC estimator, and its importance in building regression models in the face of heteroskedasticity — In this article, we’ll bring together two fundamental topics in statistical modeling, namely the covariance matrix and heteroskedasticity. Covariance matrices are the work horses of statistical inference. They are used for determining if regression coefficients are statistically significant (i.e. different from zero), and for constructing confidence intervals for each coefficient…

What happens when regression variables or the dependent variable contains measurement errors — Measurement errors can seep into an experiment for a variety of reasons such as by the measuring instrument itself, the format of the experiment, or erroneous responses submitted by survey respondents. One may argue that more often than not a variable will be measured with some error. …

We’ll learn how to use the 2SLS technique to estimate linear models containing Instrumental Variables — In this article, we’ll learn about two different ways to estimate a linear model using the Instrumental Variables technique. In the previous article, we learnt about Instrumental Variables, what they are, and when and how to use them. Let’s recap what we learnt: Consider the following linear model:

We’ll learn about Instrumental Variables, and how to use them for estimating a linear regression model — In this article, we’ll learn about an ingenious technique for estimating linear regression models using an artifact called the Instrumental Variable. Instrumental Variables (IV) based estimation allows the regression modeler to deal with a serious situation that bedevils a large fraction of regression models, one where one or more regression…

How to use proxies in place of variables that cannot be observed — Sometimes, one or more explanatory variables in a regression model cannot be observed or measured. In this article, we’ll look at how to use proxy variables in place of such unmeasurable variables. We’ll explain what makes for a good versus a bad proxy. …

While exogeneity is a good thing, endogeneity can put into question your model’s effectiveness — In this article, we’ll look at what exogenous and endogenous variables are in the context of regression analysis. We’ll also explain what happens to your regression model when one or more regression variables turn out to be endogenous. …

We’ll look at how to predict the median and other quantile points — In a regression model, one is normally interested in estimating the conditional mean of the response variable. For example, consider the following plot of vehicle prices versus number of cylinders: