Which regression coefficients pass the t test

Almost always, that is that the slope is actually horizontal so the numerical value for the slope beta is 0. The slope fit from your data is not 0. Is that discrepancy due to random chance or due to the null hypothesis being wrong? You can't ever answer that for sure, but a P value is one way to sort-of-kind-of get at an answer. The regression program reports a standard error of the slope. Compute the t ratio as the slope divided by its standard error.

Actually, it is slope minus null hypothesis slope divided by the standard error, but the null hypothesis slope is nearly always zero. Now you have a t ratio. The number of degrees of freedom df equals the number of data points minus the number of parameters fit by the regression two for linear regression.

It is essentially a one-sample t-test, comparing an observed computed value the slope with a hypothetical value the null hypothesis. The coefficient estimates the effect of the corresponding IV on the DV; the standard error of that coefficient estimates the average error in that coefficient's estimates; the t-test tells you how many times larger the coefficient itself is than the average error of the values it estimates.

The t-test tells us how many times larger the coefficient is from that error. This is consistent with other applications of a t-test; a t-test of two samples of data tells you how many times larger the difference between the sample groups' means are than the variation within the samples.

Sign up to join this community. The best answers are voted up and rise to the top. Stack Overflow for Teams — Collaborate and share knowledge with a private group. Create a free Team What is Teams? Learn more. Understanding t-test for linear regression Ask Question.

Regression analysis is based upon a functional relationship among variables and further, assumes that the relationship is linear. This linearity assumption is required because, for the most part, the theoretical statistical properties of non-linear estimation are not well worked out yet by the mathematicians and econometricians.

This presents us with some difficulties in economic analysis because many of our theoretical models are nonlinear. The marginal cost curve, for example, is decidedly nonlinear as is the total cost function, if we are to believe in the effect of specialization of labor and the Law of Diminishing Marginal Product.

There are techniques for overcoming some of these difficulties, exponential and logarithmic transformation of the data for example, but at the outset we must recognize that standard ordinary least squares OLS regression analysis will always use a linear function to estimate what might be a nonlinear relationship. This equation is the theoretical population equation and therefore uses Greek letters.

The equation we will estimate will have the Roman equivalent symbols. This is parallel to how we kept track of the population parameters and sample parameters before. The equation that will be estimated with a sample of data for two independent variables will thus be:. As with our earlier work with probability distributions, this model works only if certain assumptions hold. These are that the Y is normally distributed, the errors are also normally distributed with a mean of zero and a constant standard deviation, and that the error terms are independent of the size of X and independent of each other.

Each of these assumptions needs a bit more explanation. If one of these assumptions fails to be true, then it will have an effect on the quality of the estimates. Some of the failures of these assumptions can be fixed while others result in estimates that quite simply provide no insight into the questions the model is trying to answer or worse, give biased estimates. Figure shows the case where the assumptions of the regression model are being satisfied. The estimated line is Three values of X are shown.

A normal distribution is placed at each point where X equals the estimated line and the associated error at each value of Y. Notice that the three distributions are normally distributed around the point on the line, and further, the variation, variance, around the predicted value is constant indicating homoscedasticity from assumption 2. Figure does not show all the assumptions of the regression model, but it helps visualize these important ones.

This is the general form that is most often called the multiple regression model. Simple regression is just a special case of multiple regression. There is some value in beginning with simple regression: it is easy to graph in two dimensions, difficult to graph in three dimensions, and impossible to graph in more than three dimensions.

Consequently, our graphs will be for the simple regression case. Figure presents the regression problem in the form of a scatter plot graph of the data set where it is hypothesized that Y is dependent upon the single independent variable X.

A basic relationship from Macroeconomic Principles is the consumption function. This was called cross-section data earlier; observations on variables at one point in time across different people or other units of measurement. This analysis is often done with time series data, which would be the consumption and income of one individual or country at different points in time.

For macroeconomic problems it is common to use times series aggregated data for a whole country. The regression problem comes down to determining which straight line would best represent the data in Figure. This figure shows the assumed relationship between consumption and income from macroeconomic theory. Here the data are plotted as a scatter plot and an estimated straight line has been drawn.

From this graph we can see an error term, e 1. Each data point also has an error term. Again, the error term is put into the equation to capture effects on consumption that are not caused by income changes.

We will see how by minimizing the sum of these errors we can get an estimate for the slope and intercept of this line. Consider the graph below. The notation has returned to that for the more general model rather than the specific case of the Macroeconomic consumption function in our example.

In Figure represents the estimated value of consumption because it is on the estimated line. It is the value of y obtained using the regression line. It is not an error in the sense of a mistake. The error term was put into the estimating equation to capture missing variables and errors in measurement that may have occurred in the dependent variables.

The absolute value of a residual measures the vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line as can be seen on the graph at point X 0. If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates that actual data value for y.

In the graph, is the residual for the point shown. Here the point lies above the line and the residual is positive. Each e is a vertical distance. Using calculus, you can determine the straight line that has the parameter values of b 0 and b 1 that minimizes the SSE. When you make the SSE a minimum, you have determined the points that are on the line of best fit. It turns out that the line of best fit has the equation:.

The sample means of the x values and the y values are and , respectively. The best fit line always passes through the point , called the points of means.

These equations are called the Normal Equations and come from another very important mathematical finding called the Gauss-Markov Theorem without which we could not do regression analysis. The Gauss-Markov Theorem tells us that the estimates we get from using the ordinary least squares OLS regression method will result in estimates that have some very important properties.

Best is the statistical property that an estimator is the one with the minimum variance. Linear refers to the property of the type of line being estimated. An unbiased estimator is one whose estimating function has an expected mean equal to the mean of the population.

This is exactly the same concept here. Both Gauss and Markov were giants in the field of mathematics, and Gauss in physics too, in the 18 th century and early 19 th century. The extensive applied value of this theorem had to wait until the middle of this last century. Using the OLS method we can now find the estimate of the error variance which is the variance of the squared errors, e 2. This is sometimes called the standard error of the estimate.

This is really just the variance of the error terms and follows our regular variance formula. One important note is that here we are dividing by , which is the degrees of freedom. The degrees of freedom of a regression equation will be the number of observations, n, reduced by the number of estimated parameters, which includes the intercept as a parameter.

The variance of the errors is fundamental in testing hypotheses for a regression. As we will see shortly, the greater the dispersion about the line, meaning the larger the variance of the errors, the less probable that the hypothesized independent variable will be found to have a significant effect on the dependent variable. In short, the theory being tested will more likely fail if the variance of the error term is high. Upon reflection this should not be a surprise.

As we tested hypotheses about a mean we observed that large variances reduced the calculated test statistic and thus it failed to reach the tail of the distribution.

In those cases, the null hypotheses could not be rejected. If we cannot reject the null hypothesis in a regression problem, we must conclude that the hypothesized independent variable has no effect on the dependent variable. A way to visualize this concept is to draw two scatter plots of x and y data along a predetermined line. The first will have little variance of the errors, meaning that all the data points will move close to the line. Now do the same except the data points will have a large estimate of the error variance, meaning that the data points are scattered widely along the line.

Clearly the confidence about a relationship between x and y is effected by this difference between the estimate of the error variance. The whole goal of the regression analysis was to test the hypothesis that the dependent variable, Y, was in fact dependent upon the values of the independent variables as asserted by some foundation theory, such as the consumption function example.

Looking at the estimated equation under Figure , we see that this amounts to determining the values of b 0 and b 1. Notice that again we are using the convention of Greek letters for the population parameters and Roman letters for their estimates. The issue is how good are these estimates? In order to test a hypothesis concerning any estimate, we have found that we need to know the underlying sampling distribution.

It should come as no surprise at his stage in the course that the answer is going to be the normal distribution. If the error term is normally distributed and the variance of the estimates of the equation parameters, b 0 and b 1 , are determined by the variance of the error term, it follows that the variances of the parameter estimates are also normally distributed.

And indeed this is just the case. This hypothesis would be stated formally as:. If we cannot reject the null hypothesis, we must conclude that our theory has no validity. Therefore the effect of Income on Consumption is zero. There is no relationship as our theory had suggested. If the regression model is such that the resulting fitted regression line passes through all of the observations, then you would have a "perfect" model see a of the figure below.

In this case the model would explain all of the variability of the observations. Based on the preceding discussion of ANOVA, a perfect regression model exists when the fitted regression line passes through all observed points.

However, this is not usually the case, as seen in b of the following figure. In both of these plots, a number of points do not follow the fitted regression line.

This indicates that a part of the total variability of the observed data still remains unexplained. The error sum of squares can be obtained as the sum of squares of these deviations:. The total variability of the observed data i. The above equation is also referred to as the analysis of variance identity and can be expanded as follows:. As mentioned previously, mean squares are obtained by dividing the sum of squares by the respective degrees of freedom.

The analysis of variance approach to test the significance of regression can be applied to the yield data in the preceding table. The sum of squares can be calculated as shown next. The total sum of squares can be calculated as:. The critical value at a significance level of 0. Assuming that the desired significance is 0.

Using this result along with the scatter plot of the above figure , it can be concluded that the relationship that exists between temperature and yield is linear.

In the case of multiple linear regression models these tables are expanded to allow tests on individual variables used in the model. This is done using extra sum of squares. Multiple linear regression models and the application of extra sum of squares in the analysis of these models are discussed in Multiple Linear Regression Analysis. A confidence interval represents a closed interval where a certain percentage of the population is likely to lie. This section discusses confidence intervals used in simple linear regression analysis.

For the data in the preceding table , assume that a new value of the yield is observed after the regression model is fit to the data.

This new observation is independent of the observations used to obtain the regression model. The prediction interval values calculated in this example are shown in the figure below as Low Prediction Interval and High Prediction Interval, respectively.

It is important to analyze the regression model before inferences based on the model are undertaken. The following sections present some techniques that can be used to check the appropriateness of the model for the given data.

These techniques help to determine if any of the model assumptions have been violated. The coefficient of determination is a measure of the amount of variability in the data accounted for by the regression model. The coefficient of determination is the ratio of the regression sum of squares to the total sum of squares. These values measure different aspects of the adequacy of the regression model.

The values of S, R-sq and R-sq adj indicate how well the model fits the observed data. Plots of residuals are used to check for the following:. Examples of residual plots are shown in the following figure. Such a plot indicates an appropriate regression model. Such a plot indicates increase in variance of residuals and the assumption of constant variance is violated here.

If the residuals follow the pattern of c or d , then this is an indication that the linear regression model is not adequate. A plot of residuals may also show a pattern as seen in e , indicating that the residuals increase or decrease as the run order sequence or time progresses.