Regression Inference & Residuals
Extend simple OLS with an intercept using coefficient standard errors, t/F inference, confidence intervals and residual plots.
Question, try, then observe
A clear fitted line: a reliable model?
Illustrative output X and cost Y. Change the generating slope, noise or endpoint outlier; trace residual segments and read coefficient inference. The generating and estimated slopes need not coincide.
What did you just see?
Two equally steep lines can have observations tightly clustered around one and widely dispersed around the other. Equal slopes do not imply equal SEs or p-values. One point far along the X axis can shift the whole line.
Read the residuals
Segments connect each observation to its fitted value at the same X. The second plot places fitted values on the horizontal axis and residuals on the vertical axis. Curvature may suggest a wrong functional form; a funnel may suggest changing variance. The display itself performs no formal diagnostic test.
Assumptions of conventional inference
The conditional mean should be linear, errors should have zero conditional mean given X, and observations should be independent with equal error variance. Normal errors justify exact small-sample t/F inference. Financial series often have heavy tails, dependence and clustered volatility; a significant slope alone does not resolve these issues.
When F equals t²
With the same data, one slope, an intercept, conventional OLS assumptions and H₀:β₁=0, the overall F equals the squared slope t, and right-tail F agrees with two-sided t. This does not extend to every regression, directional test or standard-error method.
Beyond fitting
R² is the share of sample variation explained by the fit, not out-of-sample forecast accuracy. Prediction needs new-data validation and care with extrapolation; causation needs a study design and treatment of confounding. This tool adds no multiple regression, causal identification or formal time-series diagnostics.
Another example
- Enter X/Y in their original pair order; use at least three pairs with variation in X.
- Set α and fit OLS with an intercept; read coefficients, intervals, R² and t/F.
- Inspect fitted and residual plots. Check constant variance, independence and model form; significance does not establish causation.
Show formal definition and formula
Simple regression inference estimates coefficient uncertainty and tests a population slope in an OLS model with one predictor and an intercept. Diagnostic plots can reveal possible model problems; they do not automatically certify the model.
Model and residual standard error
Slope test and coefficient interval
Fit and overall test
- β₀, β₁, b₀, b₁
- Population coefficients and intercept OLS estimates.
- ŷᵢ, eᵢ
- Fitted value and residual yᵢ−ŷᵢ.
- SSE, SSR, SST
- Residual, model and total sums of squares.
- RSE, SE(bⱼ)
- Residual standard error and conventional OLS coefficient SE.
- ν, t, F
- ν=n−2; F=t² for the same zero-slope test.
- ε
- Error; conventional inference requires independence, constant variance and appropriate normality.
Common mistakes
Fit, statistical significance, prediction and causation are four different questions. A high R² does not guarantee a correct model; a small p-value does not prove X causes Y; an apparently random residual plot is not a formal certificate of every assumption.
Frequently asked questions
Are robust or Newey–West standard errors included?
No. This version uses conventional constant-variance OLS standard errors, without heteroskedasticity-robust, HAC or Newey–West estimates. Heteroskedasticity, serial dependence or volatility clustering can invalidate nominal p-values and intervals.
Does a perfect straight line give p=0?
With no residual variation the SE is zero, so a finite t ratio is undefined. The tool preserves fit information and marks undefined inference N/A instead of inventing Infinity or an exact p=0.
Is an outlier always influential?
A large residual means Y is far from the fitted line; leverage concerns an X position far from other X values. Influence also depends on how much removing that point changes the fit. They are not synonyms.
You can now explore…
- Separate fit from coefficient significance
- Use residuals to examine conventional OLS assumptions