Dlaczego

Uwaga: = suma kwadratów ogółem, = suma kwadratów błędów, a = regresja suma kwadratów. Równanie w tytule jest często zapisywane jako: $SST$ $SSE$ $SSR$

\sum_{i = 1}^{n} (y_{i} - \bar{y})^{2} = \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i})^{2} + \sum_{i = 1}^{n} ({\hat{y}}_{i} - \bar{y})^{2}

$\sum_{i=1}^n (y_i-\bar y)^2=\sum_{i=1}^n (y_i-\hat y_i)^2+\sum_{i=1}^n (\hat y_i-\bar y)^2$

Dość proste pytanie, ale szukam intuicyjnego wyjaśnienia. Intuicyjnie wydaje mi się, że miałoby większy sens. Załóżmy na przykład, że punkt ma odpowiednią wartość y i , gdzie jest odpowiednim punktem na linii regresji. Załóżmy również, że średnia wartość y dla zestawu danych wynosi . Następnie dla tego konkretnego punktu i, , podczas gdy i . Oczywiście . Czy ten wynik nie uogólniałby na cały zestaw danych? Nie rozumiem $SST\geq SSE+SSR$ $x_i$ $y_i=5$ $\hat y_i=3$ $\hat y_i$ $\bar y=0$ $SST=(5-0)^2=5^2=25$ $SSE=(5-3)^2=2^2=4$ $SSR=(3-0)^2=3^2=9$ $9+4<25$

regression least-squares r-squared Krzywka
źródło

Bardzo blisko powiązane wątki mają również dobre odpowiedzi: stats.stackexchange.com/questions/1447 , stats.stackexchange.com/questions/118 , stats.stackexchange.com/questions/123651 , stats.stackexchange.com/questions/204930 i stats.stackexchange.com/questions/127598 .

whuber

Odpowiedzi:

Dodanie i odjęcie daje Musimy więc wykazać, że

\begin{array}{rcl} \sum_{i = 1}^{n} (y_{i} - \bar{y})^{2} & = & \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i} + {\hat{y}}_{i} - \bar{y})^{2} \\ = & \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i})^{2} + 2 \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i}) ({\hat{y}}_{i} - \bar{y}) + \sum_{i = 1}^{n} ({\hat{y}}_{i} - \bar{y})^{2} \end{array}

$\begin{eqnarray*} \sum_{i=1}^n (y_i-\bar y)^2&=&\sum_{i=1}^n (y_i-\hat y_i+\hat y_i-\bar y)^2\\ &=&\sum_{i=1}^n (y_i-\hat y_i)^2+2\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y)+\sum_{i=1}^n(\hat y_i-\bar y)^2 \end{eqnarray*}$

. Zapisu

\sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i}) ({\hat{y}}_{i} - \bar{y}) = 0

$\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y)=0$

tak, (a) reszty

musi być ortogonalny do wartości zamontowane,

, i (b) suma wartości dopasowanych potrzeby, aby była równa sumie zmienną zależną,

\sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i}) ({\hat{y}}_{i} - \bar{y}) = \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i}) {\hat{y}}_{i} - \bar{y} \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i})

$\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y)=\sum_{i=1}^n(y_i-\hat y_i)\hat y_i-\bar y\sum_{i=1}^n(y_i-\hat y_i)$

e_{i} = y_{i} - {\hat{y}}_{i}

$e_i=y_i-\hat y_i$

\sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i}) {\hat{y}}_{i} = 0

$\sum_{i=1}^n(y_i-\hat y_i)\hat y_i=0$

\sum_{i = 1}^{n} y_{i} = \sum_{i = 1}^{n} {\hat{y}}_{i}

$\sum_{i=1}^ny_i=\sum_{i=1}^n\hat y_i$

Faktycznie, że (a) jest łatwiej wykazać w notacji macierzowej o ogólnym regresji wielokrotnej, której jednym przypadku zmiennej jest przypadkiem specjalnym: W przypadku (b) pochodną funkcji kryterium OLS względem stałej (! Więc potrzebny do regresji to prawda), znany jako normalne równania , jest

\begin{array}{rcl} e^{'} X \hat{β} & = & (y - X \hat{β})^{'} X \hat{β} \\ = & (y - X (X^{'} X)^{- 1} X^{'} y)^{'} X \hat{β} \\ = & y^{'} (X - X (X^{'} X)^{- 1} X^{'} X) \hat{β} \\ = & y^{'} (X - X) \hat{β} = 0 \end{array}

$\begin{eqnarray*} e'X\hat\beta &=&(y-X\hat\beta)'X\hat\beta\\ &=&(y-X(X'X)^{-1}X'y)'X\hat\beta\\ &=&y'(X-X(X'X)^{-1}X'X)\hat\beta\\ &=&y'(X-X)\hat\beta=0 \end{eqnarray*}$

które można przekształcić do

po prawej stronie w równaniu oczywiście jest również

, a

\frac{\partial S S R}{\partial \hat{α}} = - 2 \sum_{i} (y_{i} - \hat{α} - \hat{β} x_{i}) = 0,

$\frac{\partial SSR}{\partial\hat\alpha}=-2\sum_i(y_i-\hat\alpha-\hat\beta x_i)=0,$

\sum_{i} y_{i} = n \hat{α} + \hat{β} \sum_{i} x_{i}

$\sum_i y_i=n\hat\alpha+\hat\beta\sum_ix_i$

\sum_{i = 1}^{n} {\hat{y}}_{i}

$\sum_{i=1}^n\hat y_i$

{\hat{y}}_{i} = \hat{α} + \hat{β} x_{i}

$\hat y_i=\hat\alpha+\hat\beta x_i$

Christoph Hanck
źródło

$SST = SSR + SSE$

$SST$ $SST = SSR + SSE$ (exact equality).

(2) Geometric intuition

Please see the first few pictures here (especially the third): https://sites.google.com/site/modernprogramevaluation/variance-and-bias

Some of the total variation in the data (distance from datapoint to $\bar{Y}$ ) is captured by the regression line (the distance from the regression line to $\bar{Y}$ ) and error (distance from the point to the regression line). There's not room left for $SST$ to be greater than $SSE + SSR$ .

(3) The problem with your illustration

You can't look at SSE and SSR in a pointwise fashion. For a particular point, the residual may be large, so that there is more error than explanatory power from X. However, for other points, the residual will be small, so that the regression line explains a lot of the variability. They will balance out and ultimately $SST = SSR + SSE$ . Of course this is not rigorous, but you can find proofs like the above.

Also notice that regression will not be defined for one point: $b_1 = \frac{\sum(X_i -\bar{X})(Y_i-\bar{Y}) }{\sum (X_i -\bar{X})^2}$ , and you can see that the denominator will be zero, making estimation undefined.

Hope this helps.

--Ryan M.

RMurphy
źródło

When an intercept is included in linear regression(sum of residuals is zero), $SST=SSE+SSR$ .

prove

\begin{array}{rcl} S S T & = & \sum_{i = 1}^{n} (y_{i} - \bar{y})^{2} \\ = & \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i} + {\hat{y}}_{i} - \bar{y})^{2} \\ = & \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i})^{2} + 2 \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i}) ({\hat{y}}_{i} - \bar{y}) + \sum_{i = 1}^{n} ({\hat{y}}_{i} - \bar{y})^{2} \\ = & S S E + S S R + 2 \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i}) ({\hat{y}}_{i} - \bar{y}) \end{array}

$\begin{eqnarray*} SST&=&\sum_{i=1}^n (y_i-\bar y)^2\\&=&\sum_{i=1}^n (y_i-\hat y_i+\hat y_i-\bar y)^2\\&=&\sum_{i=1}^n (y_i-\hat y_i)^2+2\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y)+\sum_{i=1}^n(\hat y_i-\bar y)^2\\&=&SSE+SSR+2\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y) \end{eqnarray*}$ Just need to prove last part is equal to 0:

\begin{array}{rcl} \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i}) ({\hat{y}}_{i} - \bar{y}) & = & \sum_{i = 1}^{n} (y_{i} - β_{0} - β_{1} x_{i}) (β_{0} + β_{1} x_{i} - \bar{y}) \\ = & (β_{0} - \bar{y}) \sum_{i = 1}^{n} (y_{i} - β_{0} - β_{1} x_{i}) + β_{1} \sum_{i = 1}^{n} (y_{i} - β_{0} - β_{1} x_{i}) x_{i} \end{array}

$\begin{eqnarray*} \sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y)&=&\sum_{i=1}^n(y_i-\beta_0-\beta_1x_i)(\beta_0+\beta_1x_i-\bar y)\\&=&(\beta_0-\bar y)\sum_{i=1}^n(y_i-\beta_0-\beta_1x_i)+\beta_1\sum_{i=1}^n(y_i-\beta_0-\beta_1x_i)x_i \end{eqnarray*}$ In Least squares regression, the sum of the squares of the errors is minimized.

S S E = \sum_{i = 1}^{n} {(e_{i})}^{2} = \sum_{i = 1}^{n} {(y_{i} - \hat{y_{i}})}^{2} = \sum_{i = 1}^{n} {(y_{i} - β_{0} - β_{1} x_{i})}^{2}

$SSE=\displaystyle\sum\limits_{i=1}^n \left(e_i \right)^2= \sum_{i=1}^n\left(y_i - \hat{y_i} \right)^2= \sum_{i=1}^n\left(y_i -\beta_0- \beta_1x_i\right)^2$ Take the partial derivative of SSE with respect to

β_{0}

$\beta_0$ and setting it to zero.

\frac{\partial S S E}{\partial β_{0}} = \sum_{i = 1}^{n} 2 {(y_{i} - β_{0} - β_{1} x_{i})}^{1} = 0

$\frac{\partial{SSE}}{\partial{\beta_0}} = \sum_{i=1}^n 2\left(y_i - \beta_0 - \beta_1x_i\right)^1 = 0$ So

\sum_{i = 1}^{n} {(y_{i} - β_{0} - β_{1} x_{i})}^{1} = 0

$\sum_{i=1}^n \left(y_i - \beta_0 - \beta_1x_i\right)^1 = 0$ Take the partial derivative of SSE with respect to

β_{1}

$\beta_1$ and setting it to zero.

\frac{\partial S S E}{\partial β_{1}} = \sum_{i = 1}^{n} 2 {(y_{i} - β_{0} - β_{1} x_{i})}^{1} x_{i} = 0

$\frac{\partial{SSE}}{\partial{\beta_1}} = \sum_{i=1}^n 2\left(y_i - \beta_0 - \beta_1x_i\right)^1 x_i = 0$ So

\sum_{i = 1}^{n} {(y_{i} - β_{0} - β_{1} x_{i})}^{1} x_{i} = 0

$\sum_{i=1}^n \left(y_i - \beta_0 - \beta_1x_i\right)^1 x_i = 0$ Hence,

\sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i}) ({\hat{y}}_{i} - \bar{y}) = (β_{0} - \bar{y}) \sum_{i = 1}^{n} (y_{i} - β_{0} - β_{1} x_{i}) + β_{1} \sum_{i = 1}^{n} (y_{i} - β_{0} - β_{1} x_{i}) x_{i} = 0

$\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y)=(\beta_0-\bar y)\sum_{i=1}^n(y_i-\beta_0-\beta_1x_i)+\beta_1\sum_{i=1}^n(y_i-\beta_0-\beta_1x_i)x_i=0$

S S T = S S E + S S R + 2 \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i}) ({\hat{y}}_{i} - \bar{y}) = S S E + S S R

$SST=SSE+SSR+2\sum_{i=1}^n(y_i-\hat y_i)(\hat y_i-\bar y)=SSE+SSR$

DavidCruise
źródło

This is just the Pythagorean theorem! enter image description here

user0
źródło

stats.stackexchange.com/q/71620/171583, stats.stackexchange.com/a/256532/171583.

ayorgo