1 General Prediction

1.1 What Is Prediction?

Prediction uses information that is available now to estimate an outcome that is unknown, unobserved, or not yet observed.

Examples include:

predicting tomorrow’s temperature from recent weather measurements;
predicting a student’s exam score from earlier assignments;
predicting whether a customer will renew a subscription; and
predicting which of several image categories an object belongs to.

The general notation is

\[ \widehat{Y} = \hat f(X), \]

where \(Y\) is the outcome, \(X\) is the available information, and \(\hat f\) is a prediction rule learned from data.

Learning Goals

This note first develops prediction without causal language. We study two frameworks: prediction of a continuous outcome and prediction of a categorical outcome. Only after those foundations are clear do we apply prediction to outcome-regression modeling.

The form of the prediction depends on the type of outcome.

Outcome type	Example	Natural prediction
Continuous	Exam score, blood pressure, price	A number on the outcome scale
Binary categorical	Pass/fail, disease/no disease	A probability for the event
Multicategory	Red/green/blue, diagnosis A/B/C	One probability for each category

1.2 The Prediction Workflow

Although prediction methods differ, the basic workflow is stable.

Define the outcome and the information available at prediction time.
Use training data to learn a prediction rule.
Apply the rule to new feature values.
Compare predictions with observed outcomes.
Improve or simplify the rule based on new-data performance.

1.3 Training Data, Test Data, and Generalization

A model should predict observations it has not already seen. We therefore distinguish:

training data, used to estimate the prediction rule; and
test data, held aside and used to evaluate the fitted rule.

Performance measured on the training data is usually optimistic. Test performance better represents generalization: how well the learned pattern transfers to new observations from a similar population.

When the dataset is small, \(K\)-fold cross-validation repeatedly holds out different portions of the data. Each observation is evaluated using a model that was not trained on that observation.

2 Framework 1: Continuous-Outcome Prediction

2.1 The Target Is a Number

Suppose the outcome \(Y\) is continuous. Examples include test score, height, blood pressure, cost, or time.

The most common target is the conditional mean:

\[ f^*(x) = E(Y \mid X=x). \]

For a new case with \(X=x\), the prediction \(\hat f(x)\) is a point on the same scale as \(Y\). If the outcome is an exam score, the prediction is an exam score—not a probability.

2.2 Visual Example: Predicting an Exam Score

The following artificial dataset is used only to explain prediction. It has no causal interpretation. Study hours are the feature \(X\), and exam score is the continuous outcome \(Y\).

set.seed(20260717)

n_continuous <- 160
continuous_data <- data.frame(
  study_hours = runif(n_continuous, 0, 10)
)

continuous_data$exam_score <-
  48 + 7 * continuous_data$study_hours -
  0.35 * continuous_data$study_hours^2 +
  rnorm(n_continuous, mean = 0, sd = 5)

training_rows <- sample(
  seq_len(n_continuous),
  size = floor(0.70 * n_continuous),
  replace = FALSE
)

continuous_train <- continuous_data[training_rows, ]
continuous_test <- continuous_data[-training_rows, ]

data.frame(
  Sample = c("Training", "Test"),
  N = c(nrow(continuous_train), nrow(continuous_test)),
  Mean_score = round(c(
    mean(continuous_train$exam_score),
    mean(continuous_test$exam_score)
  ), 1)
)

##     Sample   N Mean_score
## 1 Training 112       71.8
## 2     Test  48       67.8

The points do not lie on a perfect curve. Prediction learns the systematic pattern while recognizing that individual outcomes also vary around that pattern.

2.3 Fit Prediction Rules

A linear regression rule has the form

\[ \hat Y = \hat\beta_0 + \hat\beta_1 X. \]

A curved pattern can be represented by adding a squared term:

\[ \hat Y = \hat\beta_0 + \hat\beta_1 X + \hat\beta_2 X^2. \]

linear_model <- lm(
  exam_score ~ study_hours,
  data = continuous_train
)

curved_model <- lm(
  exam_score ~ study_hours + I(study_hours^2),
  data = continuous_train
)

continuous_test$linear_prediction <- predict(
  linear_model,
  newdata = continuous_test
)

continuous_test$curved_prediction <- predict(
  curved_model,
  newdata = continuous_test
)

Each fitted line gives one predicted mean score for every value of study hours.

2.4 Prediction Errors for a Continuous Outcome

For observation \(i\), the prediction error or residual is

\[ e_i = Y_i - \hat Y_i. \]

A positive error means the observed outcome was higher than predicted. A negative error means it was lower than predicted.

Each red vertical segment is one error \(Y_i-\hat Y_i\). Shorter segments mean more accurate predictions.

Common test-set summaries are:

\[ MAE = \frac{1}{n}\sum_{i=1}^n |Y_i-\hat Y_i|, \]

\[ RMSE = \sqrt{\frac{1}{n}\sum_{i=1}^n(Y_i-\hat Y_i)^2}, \]

and

\[ R^2 = 1 - \frac{\sum_i(Y_i-\hat Y_i)^2} {\sum_i(Y_i-\bar Y)^2}. \]

MAE and RMSE are on the outcome scale. RMSE penalizes large errors more strongly. \(R^2\) compares the fitted rule with predicting the same test-set mean for everyone. Test-set \(R^2\) can be negative when the fitted rule performs worse than that simple benchmark.

linear_metrics <- regression_metrics(
  continuous_test$exam_score,
  continuous_test$linear_prediction
)

curved_metrics <- regression_metrics(
  continuous_test$exam_score,
  continuous_test$curved_prediction
)

continuous_metrics <- data.frame(
  Model = c("Linear", "Curved"),
  MAE = c(linear_metrics["MAE"], curved_metrics["MAE"]),
  RMSE = c(linear_metrics["RMSE"], curved_metrics["RMSE"]),
  R_squared = c(
    linear_metrics["R_squared"],
    curved_metrics["R_squared"]
  )
)

continuous_metrics[, -1] <- round(continuous_metrics[, -1], 3)
knitr::kable(continuous_metrics)

Model	MAE	RMSE	R_squared
Linear	4.921	6.134	0.779
Curved	3.777	4.867	0.861

Points close to the diagonal have small errors. The remaining scatter reminds us that a predicted conditional mean is not a guarantee for an individual.

2.5 Point Predictions and Prediction Intervals

A point prediction estimates the center of the outcome distribution. A prediction interval describes uncertainty for an individual future outcome.

new_student <- data.frame(study_hours = 6)

predict(
  curved_model,
  newdata = new_student,
  interval = "prediction",
  level = 0.95
)

##        fit      lwr      upr
## 1 77.77869 68.12135 87.43602

The interval is wider than uncertainty about the mean because individual outcomes vary even when the mean pattern is known.

3 Framework 2: Categorical-Outcome Prediction

3.1 The Target Is a Category Probability

For a binary outcome \(Y\in\{0,1\}\), the conditional mean is a probability:

\[ E(Y\mid X=x)=P(Y=1\mid X=x). \]

The natural prediction is therefore

\[ \hat p(x)=\widehat P(Y=1\mid X=x), \]

a number between 0 and 1. A probability of 0.70 is not yet a category assignment. It represents estimated uncertainty about the outcome.

3.2 Visual Example: Predicting Pass or Fail

We now use a second artificial dataset. The outcome is pass (\(Y=1\)) or fail (\(Y=0\)), and the feature is a practice score.

set.seed(20260718)

n_binary <- 240
binary_data <- data.frame(
  practice_score = runif(n_binary, 0, 10)
)

true_pass_probability <- plogis(
  -4 + 0.75 * binary_data$practice_score
)

binary_data$passed <- rbinom(
  n_binary,
  size = 1,
  prob = true_pass_probability
)

binary_training_rows <- sample(
  seq_len(n_binary),
  size = floor(0.70 * n_binary),
  replace = FALSE
)

binary_train <- binary_data[binary_training_rows, ]
binary_test <- binary_data[-binary_training_rows, ]

Logistic regression models the log odds of the event:

\[ \operatorname{logit}\{P(Y=1\mid X)\} =\beta_0+\beta_1X, \]

and converts the fitted value back to the probability scale:

\[ \hat p(X)= \frac{1}{1+\exp\{-(\hat\beta_0+\hat\beta_1X)\}}. \]

binary_model <- glm(
  passed ~ practice_score,
  data = binary_train,
  family = binomial()
)

binary_test$predicted_probability <- predict(
  binary_model,
  newdata = binary_test,
  type = "response"
)

The observed outcomes occupy only 0 and 1, but the fitted rule produces a smooth range of probabilities.

3.3 Evaluate Probability Predictions

Probability predictions should be evaluated as probabilities before converting them to categories.

3.3.1 Brier Score

The Brier score is squared prediction error:

\[ \text{Brier} =\frac{1}{n}\sum_{i=1}^n(Y_i-\hat p_i)^2. \]

Smaller values indicate better probability predictions.

3.3.2 Log Loss

Binary log loss is

\[ \text{Log loss} =-\frac{1}{n}\sum_{i=1}^n \left[ Y_i\log(\hat p_i) +(1-Y_i)\log(1-\hat p_i) \right]. \]

Smaller is better. Confident but incorrect predictions receive a large penalty.

3.3.3 Discrimination

Discrimination asks whether cases tend to receive higher predicted probabilities than non-cases. The area under the ROC curve (AUC) is 0.5 for random ranking and 1 for perfect ranking.

3.3.4 Calibration

Calibration asks whether predicted probabilities agree with observed frequencies. Among observations assigned a probability near 0.70, approximately 70 percent should experience the event in well-calibrated new data.

y_binary_test <- binary_test$passed
p_binary_test <- bound_probability(binary_test$predicted_probability)

binary_metrics <- data.frame(
  Metric = c("Brier score", "Log loss", "AUC"),
  Value = c(
    mean((y_binary_test - p_binary_test)^2),
    -mean(
      y_binary_test * log(p_binary_test) +
      (1 - y_binary_test) * log(1 - p_binary_test)
    ),
    auc_rank(y_binary_test, p_binary_test)
  )
)

binary_metrics$Value <- round(binary_metrics$Value, 4)
knitr::kable(binary_metrics)

Metric	Value
Brier score	0.1522
Log loss	0.4665
AUC	0.8516

calibration_group <- cut(
  rank(p_binary_test, ties.method = "first"),
  breaks = 6,
  labels = FALSE
)

binary_calibration <- aggregate(
  cbind(
    predicted = p_binary_test,
    observed = y_binary_test
  ),
  by = list(Group = calibration_group),
  FUN = mean
)

binary_calibration$N <- as.vector(table(calibration_group))

knitr::kable(
  transform(
    binary_calibration,
    predicted = round(predicted, 3),
    observed = round(observed, 3)
  )
)

Group	predicted	observed	N
1	0.072	0.083	12
2	0.281	0.250	12
3	0.591	0.500	12
4	0.763	0.667	12
5	0.867	0.917	12
6	0.946	0.917	12

3.4 From Probability to Category

Classification requires a decision threshold \(c\):

\[ \widehat Y_i = \begin{cases} 1, & \hat p_i \ge c,\\ 0, & \hat p_i < c. \end{cases} \]

The threshold is a decision choice, not a fact learned automatically from the outcome. Changing it changes sensitivity, specificity, and the numbers of false positives and false negatives.

threshold <- 0.50
predicted_class <- as.integer(p_binary_test >= threshold)

confusion_matrix <- table(
  Predicted = factor(
    predicted_class,
    levels = 0:1,
    labels = c("Fail", "Pass")
  ),
  Observed = factor(
    y_binary_test,
    levels = 0:1,
    labels = c("Fail", "Pass")
  )
)

confusion_matrix

##          Observed
## Predicted Fail Pass
##      Fail   20    5
##      Pass   12   35

sensitivity <- sum(predicted_class == 1 & y_binary_test == 1) /
  sum(y_binary_test == 1)
specificity <- sum(predicted_class == 0 & y_binary_test == 0) /
  sum(y_binary_test == 0)
accuracy <- mean(predicted_class == y_binary_test)

data.frame(
  Threshold = threshold,
  Sensitivity = round(sensitivity, 3),
  Specificity = round(specificity, 3),
  Accuracy = round(accuracy, 3)
)

##   Threshold Sensitivity Specificity Accuracy
## 1       0.5       0.875       0.625    0.764

Probability prediction and classification are related but distinct:

probability prediction estimates uncertainty;
classification turns that uncertainty into an action;
the best threshold depends on the costs of the two kinds of error.

3.5 More Than Two Categories

If \(Y\) has \(K\) unordered categories, the model predicts a probability vector:

\[ \left\{ \hat p_1(X),\hat p_2(X),\ldots,\hat p_K(X) \right\}, \qquad \sum_{k=1}^K \hat p_k(X)=1. \]

The predicted class is often the category with the largest probability. Multinomial logistic regression, classification trees, and many machine-learning methods can estimate these probabilities.

For ordered categories such as mild, moderate, and severe, models can also use the category ordering.

4 Ideas Shared by Both Frameworks

4.1 Continuous and Categorical Prediction Compared

Feature	Continuous outcome	Categorical outcome
Example	Exam score	Pass/fail
Model output	Predicted number	Predicted category probabilities
Common model	Linear regression	Logistic regression
Individual error	\(Y-\hat Y\)	Probability loss or classification error
Common metrics	MAE, RMSE, \(R^2\)	Brier score, log loss, AUC, calibration
Optional decision step	Usually none	Choose a threshold or largest probability

The common principle is to learn a function from inputs to an outcome-relevant quantity and to judge it on observations not used to learn the function.

4.2 Underfitting and Overfitting

Underfitting occurs when a prediction rule is too simple to capture an important pattern. Overfitting occurs when a rule follows random details of the training sample that do not repeat in new data.

Model behavior	Training performance	Test performance
Underfit	Poor	Poor
Useful complexity	Good	Good
Overfit	Extremely good	Worse

More predictors and more flexible curves can reduce training error without improving test error. Validation, cross-validation, regularization, and subject-matter knowledge help control complexity.

4.3 Prediction Does Not Explain Why

Prediction answers:

Given the information available, what outcome is likely?

It does not by itself answer:

Why did the outcome occur?
What would happen if one input were changed by intervention?
Is a predictor a cause, consequence, or merely a correlate of the outcome?

This distinction becomes essential when prediction is used inside causal inference.

5 Application to Outcome Regression

5.1 Where Prediction Enters Outcome Regression

We now move from general prediction to the causal inference course.

Let:

\(T\) be a binary treatment or exposure;
\(X\) be baseline covariates; and
\(Y\) be the outcome.

An outcome model estimates

\[ m(t,x)=E(Y\mid T=t,X=x). \]

For a binary outcome, this is a probability:

\[ m(t,x)=P(Y=1\mid T=t,X=x). \]

Because \(T\) is binary, the prediction function is evaluated at two treatment values:

Treatment or exposure set to 1:

\[ m(1,x) = E(Y\mid T=1,X=x) = P(Y=1\mid T=1,X=x). \]

For people with baseline covariate profile \(X=x\), \(m(1,x)\) is the conditional mean outcome. When \(Y\) is binary, it is the conditional probability of the event among those with \(T=1\).
Treatment or exposure set to 0:

\[ m(0,x) = E(Y\mid T=0,X=x) = P(Y=1\mid T=0,X=x). \]

For people with the same baseline covariate profile \(X=x\), \(m(0,x)\) is the conditional mean outcome. For binary \(Y\), it is the conditional probability of the event among those with \(T=0\).

The fitted outcome model estimates these two quantities as \(\hat m(1,x)\) and \(\hat m(0,x)\).

The outcome-regression procedure uses this prediction function in a special way: it predicts each person’s outcome under both treatment settings and then averages.

5.2 Load the Classroom NHANES Data

We use the same complete-case classroom dataset as the other lectures. The example predicts CVD from smoking status and baseline covariates. Survey weights are not used because the purpose is to explain the estimator rather than conduct a full NHANES survey analysis.

data_candidates <- c(
  file.path("..", "Data", "NHANES_data.csv"),
  file.path("Data", "NHANES_data.csv"),
  "C:/Users/seyoo/OneDrive/GitHub_repository/Causal_Inference_Survival_Analysis/Causal_Inference_Lecture/Data/NHANES_data.csv"
)

data_path <- data_candidates[file.exists(data_candidates)][1]

if (is.na(data_path)) {
  stop("Could not find NHANES_data.csv. Check the data path.")
}

nhanes <- read.csv(data_path, stringsAsFactors = FALSE)

analysis_vars <- c(
  "cvd_indicator",
  "smoker_indicator",
  "age_yr",
  "gender",
  "race",
  "educ_lvl",
  "inc_to_pov_ratio",
  "bmi"
)

or_data <- nhanes[
  complete.cases(nhanes[, analysis_vars]),
  analysis_vars
]

or_data$cvd_indicator <- as.integer(or_data$cvd_indicator)
or_data$smoker_indicator <- as.integer(or_data$smoker_indicator)
or_data$gender <- factor(or_data$gender)
or_data$race <- factor(or_data$race)
or_data$educ_lvl <- factor(or_data$educ_lvl)

data.frame(
  Observations = nrow(or_data),
  CVD_cases = sum(or_data$cvd_indicator),
  CVD_percent = round(100 * mean(or_data$cvd_indicator), 2),
  Smoker_percent = round(100 * mean(or_data$smoker_indicator), 2)
)

##   Observations CVD_cases CVD_percent Smoker_percent
## 1         6299       646       10.26          40.63

5.3 Step 1: Fit the Outcome Prediction Model

For the binary CVD outcome, we fit logistic regression:

\[ \operatorname{logit}\{m(T,X)\} =\beta_0+\beta_TT+\beta_X^\top X. \]

outcome_model <- glm(
  cvd_indicator ~ smoker_indicator + age_yr + gender + race +
    educ_lvl + inc_to_pov_ratio + bmi,
  data = or_data,
  family = binomial()
)

model_description <- data.frame(
  Component = c("Outcome", "Exposure", "Baseline covariates", "Model output"),
  Specification = c(
    "CVD indicator (0/1)",
    "Smoking indicator (0/1)",
    "Age, gender, race, education, income-to-poverty ratio, BMI",
    "Predicted conditional CVD probability"
  )
)

knitr::kable(model_description)

Component	Specification
Outcome	CVD indicator (0/1)
Exposure	Smoking indicator (0/1)
Baseline covariates	Age, gender, race, education, income-to-poverty ratio, BMI
Model output	Predicted conditional CVD probability

The model describes the observed conditional CVD probability. Its individual coefficients are not the standardized causal effect; the effect-scale calculation comes after scenario prediction and averaging.

5.4 Step 2: Predict Two Treatment Scenarios

Make two copies of the same analytic sample:

set smoking to \(T=1\) for everyone;
set smoking to \(T=0\) for everyone;
keep age, gender, race, education, income, and BMI unchanged.

data_if_smoker <- or_data
data_if_non_smoker <- or_data

data_if_smoker$smoker_indicator <- 1
data_if_non_smoker$smoker_indicator <- 0

m1_hat <- predict(
  outcome_model,
  newdata = data_if_smoker,
  type = "response"
)

m0_hat <- predict(
  outcome_model,
  newdata = data_if_non_smoker,
  type = "response"
)

scenario_examples <- data.frame(
  Age = or_data$age_yr[1:10],
  BMI = round(or_data$bmi[1:10], 1),
  Observed_smoking = or_data$smoker_indicator[1:10],
  Predicted_if_smoker = round(m1_hat[1:10], 3),
  Predicted_if_non_smoker = round(m0_hat[1:10], 3)
)

knitr::kable(scenario_examples)

	Age	BMI	Observed_smoking	Predicted_if_smoker	Predicted_if_non_smoker
1	29	37.8	0	0.007	0.005
3	36	21.9	1	0.054	0.035
5	76	26.6	1	0.323	0.237
7	33	28.9	0	0.015	0.009
8	68	28.1	0	0.111	0.075
9	58	30.5	0	0.155	0.106
10	44	30.1	0	0.132	0.090
11	54	24.9	0	0.041	0.027
13	68	34.2	0	0.446	0.343
14	54	29.6	0	0.034	0.022

These are two model predictions for each person. They are not two observed outcomes: only one smoking status was actually observed.

Interpreting one ordered pair. The highlighted point is an example person \(j\). Reading the horizontal coordinate first and the vertical coordinate second, this person’s ordered pair is

\[ \left(\hat m(0,X_j),\hat m(1,X_j)\right) = \left( 0.300, 0.397 \right). \]

The first coordinate, \(\hat m(0,X_j)=\) 0.300, is the predicted CVD risk when this person’s smoking status is set to \(T=0\). In percentage terms, that is about 30.0 percent.
The second coordinate, \(\hat m(1,X_j)=\) 0.397, is the predicted CVD risk for the same person, with the same baseline covariates \(X_j\), when smoking is set to \(T=1\). That is about 39.7 percent.
The model-based scenario contrast is \(\hat m(1,X_j)-\hat m(0,X_j)=\) 0.097, or approximately 9.7 percentage points higher in the smoking scenario.

The orange point is above the dashed equality line because its second coordinate is larger than its first. This ordered pair does not represent two outcomes observed for person \(j\), and its difference is not a directly observed individual causal effect. It contains two conditional mean predictions from the fitted model; only one smoking state and one outcome were actually observed for this person.

Causal interpretation under the identification assumptions. If consistency, conditional exchangeability given \(X\), and positivity hold—and the outcome model is adequately specified—the two fitted predictions can identify conditional potential-outcome risks:

\[ \hat m(0,X_j) \approx P\{Y(0)=1\mid X=X_j\} = 0.300, \]

and

\[ \hat m(1,X_j) \approx P\{Y(1)=1\mid X=X_j\} = 0.397. \]

Therefore,

\[ \hat m(1,X_j)-\hat m(0,X_j) \approx E\{Y(1)-Y(0)\mid X=X_j\} = 0.097. \]

For people with this baseline profile, the model estimates that setting smoking to \(1\), rather than \(0\), raises CVD risk by about 9.7 percentage points. This is a conditional average causal risk difference for the covariate profile \(X_j\), not the unobservable individual causal effect \(Y_j(1)-Y_j(0)\). The point lies above the equality line because this estimated conditional causal contrast is positive.

5.5 Step 3: Standardize the Predictions

Average each set of predicted probabilities over the same covariate distribution:

\[ \hat\psi_1^{OR} =\frac{1}{n}\sum_{i=1}^n\hat m(1,X_i), \]

\[ \hat\psi_0^{OR} =\frac{1}{n}\sum_{i=1}^n\hat m(0,X_i). \]

Then compare the standardized risks:

\[ \widehat{RD}^{OR} =\hat\psi_1^{OR}-\hat\psi_0^{OR}, \]

\[ \widehat{RR}^{OR} =\frac{\hat\psi_1^{OR}}{\hat\psi_0^{OR}}. \]

psi1_hat <- mean(m1_hat)
psi0_hat <- mean(m0_hat)
rd_hat <- psi1_hat - psi0_hat
rr_hat <- psi1_hat / psi0_hat

or_results <- data.frame(
  Quantity = c(
    "Standardized risk if everyone smoked",
    "Standardized risk if no one smoked",
    "Standardized risk difference",
    "Standardized risk ratio"
  ),
  Value = c(psi1_hat, psi0_hat, rd_hat, rr_hat)
)

or_results$Value <- round(or_results$Value, 4)
knitr::kable(or_results)

Quantity	Value
Standardized risk if everyone smoked	0.1198
Standardized risk if no one smoked	0.0853
Standardized risk difference	0.0345
Standardized risk ratio	1.4047

The key operation is average after predicting. We do not classify people as CVD or no CVD. Outcome regression retains the predicted probabilities and averages them.

5.6 Ordinary Prediction Versus Outcome Regression

Ordinary outcome prediction	Outcome-regression standardization
Insert each person’s observed \(T\) and \(X\)	Set \(T=1\) and \(T=0\) for every person
Produce one fitted risk per person	Produce two scenario-specific risks per person
Evaluate prediction of observed \(Y\)	Average each scenario over the same \(X\) distribution
Target observed conditional risk	Target marginal potential-outcome risk under assumptions

The fitted function is a prediction model in both columns. The data operation and interpretation differ.

5.7 Why Good Prediction Is Not Enough for Causality

Prediction quality cannot establish that a contrast is causal. A model may predict observed CVD well and still produce a biased intervention comparison.

For a causal interpretation, outcome regression requires:

Consistency: the observed outcome under the observed treatment equals the corresponding potential outcome.
Conditional exchangeability: after conditioning on \(X\), treatment is independent of the potential outcomes.
Positivity: each relevant covariate pattern has a positive probability of each treatment.
Adequate outcome modeling: \(\hat m(t,X)\) represents the relevant conditional outcome risks sufficiently well.

Variables that improve prediction are not automatically appropriate causal adjustment variables. A variable caused by treatment may predict the outcome well but create post-treatment adjustment bias. A collider may also improve prediction while distorting a causal comparison.

5.8 Continuous Outcomes in Outcome Regression

The same standardization logic works when \(Y\) is continuous. For example, if \(Y\) were blood pressure, a regression model would estimate

\[ m(t,x)=E(Y\mid T=t,X=x). \]

We would predict a blood-pressure value under \(T=1\) and \(T=0\) for every person and average:

\[ \hat\psi_t^{OR} =\frac{1}{n}\sum_{i=1}^n\hat m(t,X_i). \]

The only change is the scale of the prediction:

binary outcome: predicted probability;
continuous outcome: predicted conditional mean.

6 Summary

6.1 Prediction Checklist

Before interpreting a prediction analysis, ask:

What is the outcome?
Is it continuous, binary, or multicategory?
What information is available at prediction time?
What quantity does the model output?
Was performance evaluated outside the training data?
Which metric matches the outcome and intended use?
Is the model calibrated as well as discriminative?
If categories are assigned, how was the decision threshold chosen?

For causal outcome regression, also ask:

Are the adjustment variables appropriate baseline confounders?
Are consistency, exchangeability, and positivity plausible?
Does the model predict reasonably under both treatment settings?
Does the uncertainty calculation repeat the full fitting and standardization procedure?

6.2 Key Takeaways

Prediction learns a rule that maps available information to an unknown outcome.
Continuous-outcome prediction returns a number on the outcome scale and is commonly evaluated with MAE, RMSE, and \(R^2\).
Categorical-outcome prediction returns category probabilities and is evaluated with probability, calibration, discrimination, and classification measures.
Probability prediction and category assignment are separate steps.
Test data or cross-validation are needed to evaluate generalization.
Outcome regression uses prediction as a building block: predict under both treatment settings, average, and compare.
Predictive performance alone does not give a causal interpretation; causal assumptions and appropriate variable selection are still required.

6.3 Appendix: Connection to the G-Formula

Under consistency, conditional exchangeability, and positivity,

\[ E\{Y(t)\} =E_X[E(Y\mid T=t,X)]. \]

The inner expectation is the prediction function

\[ m(t,X)=E(Y\mid T=t,X), \]

and the outer expectation standardizes it over the covariate distribution. Its sample analogue is

\[ \hat\psi_t^{OR} =\frac{1}{n}\sum_{i=1}^n\hat m(t,X_i). \]

Outcome-regression standardization is therefore also called parametric g-computation. Prediction provides \(\hat m\); identification assumptions connect the observed conditional mean to the potential-outcome mean.

6.4 References

Harrell, F. E. (2015). Regression Modeling Strategies (2nd ed.). Springer.

Hernan, M. A., & Robins, J. M. (2020). Causal Inference: What If. Chapman & Hall/CRC.

James, G., Witten, D., Hastie, T., Tibshirani, R., & Taylor, J. (2023). An Introduction to Statistical Learning (2nd ed.). Springer.

Steyerberg, E. W. (2019). Clinical Prediction Models (2nd ed.). Springer.

Prediction: Foundations and Its Role in Outcome Regression

Se Yoon Lee

2026-07-17

1 General Prediction

1.1 What Is Prediction?

1.2 The Prediction Workflow

1.3 Training Data, Test Data, and Generalization

2 Framework 1: Continuous-Outcome Prediction

2.1 The Target Is a Number

2.2 Visual Example: Predicting an Exam Score

2.3 Fit Prediction Rules

2.4 Prediction Errors for a Continuous Outcome

2.5 Point Predictions and Prediction Intervals

3 Framework 2: Categorical-Outcome Prediction

3.1 The Target Is a Category Probability

3.2 Visual Example: Predicting Pass or Fail

3.3 Evaluate Probability Predictions

3.3.1 Brier Score

3.3.2 Log Loss

3.3.3 Discrimination

3.3.4 Calibration

3.4 From Probability to Category

3.5 More Than Two Categories

4 Ideas Shared by Both Frameworks

4.1 Continuous and Categorical Prediction Compared

4.2 Underfitting and Overfitting

4.3 Prediction Does Not Explain Why

5 Application to Outcome Regression

5.1 Where Prediction Enters Outcome Regression

5.2 Load the Classroom NHANES Data

5.3 Step 1: Fit the Outcome Prediction Model

5.4 Step 2: Predict Two Treatment Scenarios

5.5 Step 3: Standardize the Predictions

5.6 Ordinary Prediction Versus Outcome Regression

5.7 Why Good Prediction Is Not Enough for Causality

5.8 Continuous Outcomes in Outcome Regression

6 Summary

6.1 Prediction Checklist

6.2 Key Takeaways

6.3 Appendix: Connection to the G-Formula

6.4 References