Download presentation

Presentation is loading. Please wait.

1
**Statistics for the Social Sciences**

Psychology 340 Spring 2005 Prediction

2
Outline (for week) Simple bi-variate regression, least-squares fit line The general linear model Residual plots Using SPSS Multiple regression Comparing models, (?? Delta r2)

3
Regression Last time: with correlation, we examined whether variables X & Y are related This time: with regression, we try to predict the value of one variable given what we know about the other variable and the relationship between the two.

4
Regression Last time: “it doesn’t matter which variable goes on the X-axis or the Y-axis” The variable that you are predicting goes on the Y-axis (criterion variable) Predicted variable For regression this is NOT the case Y X 1 2 3 4 5 6 Quiz performance Predicting variable The variable that you are making the prediction based on goes on the X-axis (predictor variable) Hours of study

5
**Regression Last time: “Imagine a line through the points”**

But there are lots of possible lines Y X 1 2 3 4 5 6 One line is the “best fitting line” Today: learn how to compute the equation corresponding to this “best fitting line” Quiz performance Hours of study

6
**The equation for a line A brief review of geometry**

Y = intercept, when X = 0 Y X 1 2 3 4 5 6 Y = (X)(slope) + (intercept) 2.0

7
**The equation for a line A brief review of geometry**

X 1 2 3 4 5 6 Y = (X)(slope) + (intercept) 0.5 2.0 1 2 Change in Y Change in X = slope

8
**The equation for a line A brief review of geometry**

X 1 2 3 4 5 6 Y = (X)(slope) + (intercept) Y = (X)(0.5) + 2.0

9
**Regression A brief review of geometry Consider a perfect correlation**

X = 5 Y = ? Y X 1 2 3 4 5 6 Y = (X)(0.5) + (2.0) Y = (5)(0.5) + (2.0) Y = = 4.5 4.5 Can make specific predictions about Y based on X

10
**Regression Consider a less than perfect correlation**

The line still represents the predicted values of Y given X X = 5 Y = ? Y X 1 2 3 4 5 6 Y = (X)(0.5) + (2.0) Y = (5)(0.5) + (2.0) Y = = 4.5 4.5

11
Regression The “best fitting line” is the one that minimizes the error (differences) between the predicted scores (the line) and the actual scores (the points) Y X 1 2 3 4 5 6 Rather than compare the errors from different lines and picking the best, we will directly compute the equation for the best fitting line

12
**Regression The linear model Y = intercept + slope (X) + error**

Beta’s () are sometimes called parameters Come in two types: standardized unstanderdized Now let’s go through an example computing these things

13
**Scatterplot Using the dataset from our correlation lecture Y X Y 6 6**

6 5 4 3 2 1 X 1 2 3 4 5 6

14
**From the Computing Pearson’s r lecture**

mean 3.6 4.0 2.4 -2.6 1.4 -0.6 0.0 2.0 -2.0 4.8 5.2 2.8 1.2 5.76 6.76 1.96 0.36 X Y 14.0 15.20 16.0 SSY SSX SP

15
**Computing regression line (with raw scores)**

X Y mean 3.6 4.0 14.0 15.20 16.0 SSY SSX SP

16
**Computing regression line (with raw scores)**

Y X 1 2 3 4 5 6 X Y mean 3.6 4.0

17
**Computing regression line (with raw scores)**

Y X Y 6 5 4 The two means will be on the line 3 2 1 mean 3.6 4.0 X 1 2 3 4 5 6

18
**Computing regression line (standardized, using z-scores)**

Sometimes the regression equation is standardized. Computed based on z-scores rather than with raw scores X Y 2.4 5.76 2.0 4.0 1.38 1.1 -2.6 6.76 -2.0 4.0 -1.49 -1.1 1.4 1.96 2.0 4.0 0.8 1.1 -0.6 0.36 0.0 0.0 - 0.34 0.0 -0.6 0.36 -2.0 4.0 - 0.34 -1.1 Mean 3.6 4.0 0.0 15.20 0.0 16.0 0.0 0.0 1.74 1.79 Std dev

19
**Computing regression line (standardized, using z-scores)**

Sometimes the regression equation is standardized. Computed based on z-scores rather than with raw scores Prediction model Predicted Z score (on criterion variable) = standardized regression coefficient multiplied by Z score on predictor variable Formula 1.38 1.1 -1.49 -1.1 0.8 1.1 - 0.34 0.0 - 0.34 -1.1 The standardized regression coefficient (β) In bivariate prediction, β = r 0.0 0.0

20
**Computing regression line (with z-scores)**

ZY 2 1 1.38 1.1 ZX -1.49 -1.1 -2 -1 1 2 -1 0.8 1.1 - 0.34 0.0 -2 - 0.34 -1.1 0.0 0.0 mean

21
**Regression The linear equation isn’t the whole thing**

Y = intercept + slope (X) + error The linear equation isn’t the whole thing Also need a measure of error Y = X(.5) + (2.0) + error Y = X(.5) + (2.0) + error Same line, but different relationships (strength difference) Y X 1 2 3 4 5 6 Y X 1 2 3 4 5 6

22
**Regression Error Actual score minus the predicted score**

Measures of error r2 (r-squared) Proportionate reduction in error Note: Total squared error when predicting from the mean = SSTotal=SSY Squared error using prediction model = Sum of the squared residuals = SSresidual= SSerror

23
**R-squared r2 represents the percent variance in Y accounted for by X**

1 2 3 4 5 6 Y X 1 2 3 4 5 6 64% variance explained 25% variance explained

24
**Computing Error around the line**

Sum of the squared residuals = SSresidual = SSerror Compute the difference between the predicted values and the observed values (“residuals”) Y X 1 2 3 4 5 6 Square the differences Add up the squared differences

25
**Computing Error around the line**

Sum of the squared residuals = SSresidual = SSerror X Y mean 3.6 Predicted values of Y (points on the line) 4.0

26
**Computing Error around the line**

Sum of the squared residuals = SSresidual = SSerror X Y 6.2 = (0.92)(6)+0.688 mean 3.6 Predicted values of Y (points on the line) 4.0

27
**Computing Error around the line**

Sum of the squared residuals = SSresidual = SSerror X Y 6.2 = (0.92)(6)+0.688 1.6 = (0.92)(1)+0.688 5.3 = (0.92)(5)+0.688 3.45 = (0.92)(3)+0.688 3.45 = (0.92)(3)+0.688 mean 3.6 4.0

28
**Computing Error around the line**

Sum of the squared residuals = SSresidual = SSerror X Y Y X 1 2 3 4 5 6 6.2 6.2 1.6 5.3 3.45 1.6 5.3 3.45 3.45

29
**Computing Error around the line**

Sum of the squared residuals = SSresidual = SSerror residuals X Y 6.2 = -0.20 1.6 = 0.40 5.3 = 0.70 3.45 = 0.55 3.45 -1.45 = mean 3.6 4.0 Quick check 0.00

30
**Computing Error around the line**

Sum of the squared residuals = SSresidual = SSerror X Y 6.2 -0.20 0.04 1.6 0.40 0.16 5.3 0.70 0.49 3.45 0.55 0.30 3.45 -1.45 2.10 mean 3.6 4.0 0.00 3.09 SSERROR

31
**Computing Error around the line**

Sum of the squared residuals = SSresidual = SSerror 4.0 0.0 16.0 SSY X Y 6.2 -0.20 0.04 1.6 0.40 0.16 5.3 0.70 0.49 3.45 0.55 0.30 3.45 -1.45 2.10 mean 3.6 4.0 0.00 3.09 SSERROR

32
**Computing Error around the line**

Sum of the squared residuals = SSresidual = SSerror Proportionate reduction in error Also (like r2) represents the percent variance in Y accounted for by X In fact, it is mathematically identical to r2 16.0 3.09 SSY SSERROR

33
**Seeing patterns in the error**

Residual plots The sum of the residuals should always equal 0 (as should the mean). the least squares regression line splits the data in half, half of the error is above the line and half is below the line. In addition to summing to zero, we also want there the residuals to be randomly distributed. That is, there should be no pattern to the residuals. If there is a pattern, it may suggest that there is more than a simple linear relationship between the two variables. Residual plots are very useful tools to examine the relationship even further. These are basically scatterplots of the residuals () against the Explanatory (X) variable (note: the examples actually plot the residuals that have transformed into z-scores).

34
**Seeing patterns in the error**

Scatter plot Residual plot The scatterplot shows a nice linear relationship. The residual plot shows that the residuals fall randomly above and below the line. Critically there doesn't seem to be a discernable pattern to the residuals.

35
**Seeing patterns in the error**

Scatter plot Residual plot The residual plot shows that the residuals get larger as X increases. This suggests that the variability around the line is not constant across values of X. This is referred to as a violation of homogeniety of variance. The scatterplot also shows a nice linear relationship.

36
**Seeing patterns in the error**

Scatter plot Residual plot The scatterplot shows what may be a linear relationship. The residual plot suggests that a non-linear relationship may be more appropriate (see how a curved pattern appears in the residual plot).

37
**Regression in SPSS Running the analysis is pretty easy**

Analyze: Regression: Linear Predictor variables go into the ‘independent variable’ field (Predicted variable) goes into the “dependent variable’ field You get a lot of output

38
**Regression in SPSS The variables in the model r r2**

We’ll get back to these numbers in a few weeks Unstandardized coefficients Slope (indep var name) Intercept (constant) Standardized coefficients

39
**Multiple Regression Multiple regression prediction models “fit”**

“residual”

40
**Prediction in Research Articles**

Bivariate prediction models rarely reported Multiple regression results commonly reported

Similar presentations

© 2021 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google