Logistic Regression — Se Yoon Lee, Ph.D.

Build and fit the model

Generate a training cohort and an independent validation cohort, then refit logistic regression in the browser.

Scenario presets

Data-generating mechanism

Training sample size

2502,500

Baseline CVD risk

1%45%

True age OR per 10 years

True smoking OR

Age nonlinearity

nonestrong quadratic

Smoking × age interaction

common OReffect modification

Treatment-selection strength

nearly randomstrongly selected

Cohort seed

Working model and evaluation

Outcome-model specification

Standard errors

Diagnostic sample

Current subject profile

Age

BMI

Income-to-poverty ratio

Female

Smoker

Mathematical formulation

Move from a binary conditional mean to odds, maximum likelihood, Wald inference, and causal nuisance functions.

NATIVE MATHML

Binary conditional mean

E (Y | X = x) = P (Y = 1 | X = x) = p (x), logit {p (x)} = x^{T} β

Odds and log odds

odds (p) = \frac{p}{1 - p}, logit (p) = \log {\frac{p}{1 - p}}

Inverse link

expit (η) = \frac{e^{η}}{1 + e^{η}} = \frac{1}{1 + e^{- η}}

Bernoulli likelihood

L (β) = \prod_{i = 1}^{n} p_{i}^{Y_{i}} {(1 - p_{i})}^{1 - Y_{i}}, p_{i} = expit (x_{i}^{T} β)

Score and information

\begin{matrix} U (β) = X^{T} (Y - p) \\ I (β) = X^{T} W X, W_{i i} = p_{i} (1 - p_{i}) \end{matrix}

Newton–Raphson / IRLS

β^{(k + 1)} = β^{(k)} + {X^{T} W X}^{- 1} X^{T} (Y - p)

Conditional odds-ratio interpretation

\frac{odds {Y = 1 | X_{j} + 1, X_{- j}}}{odds {Y = 1 | X_{j}, X_{- j}}} = \exp (β_{j})

Probability change is not constant

Δ p = expit (η + β_{j}) - expit (η)

The same odds ratio can correspond to very different risk differences at different baseline risks.

Odds ratio is not risk ratio

RR (p, OR) = \frac{OR}{1 - p + OR \cdot p}

Wald inference

\hat{β} \approx N {β, {(X^{T} \hat{W} X)}^{- 1}}

Coefficient interval

{\hat{β}}_{j} \pm 1.96 \hat{SE} ({\hat{β}}_{j})

Odds-ratio interval

\exp [{\hat{β}}_{j} \pm 1.96 \hat{SE} ({\hat{β}}_{j})]

Two causal-inference nuisance functions

e (X) = P (T = 1 | X), m (t, X) = P (Y = 1 | T = t, X)

Outcome-regression standardization

{\hat{ψ}}_{t} = \frac{1}{n} \sum_{i = 1}^{n} \hat{m} (t, X_{i})

ATE inverse-probability weight

w_{i} = \frac{T_{i}}{\hat{e}} + \frac{1 - T_{i}}{1 - \hat{e} (X_{i})}

1. Probability, odds, and log-odds map

The logistic function maps an unbounded linear predictor to a valid probability.

expit(η)current subject

2. Current subject: additive log-odds contributions

Term contributions add on the η scale; the final inverse-logit transformation is nonlinear.

3. Binary outcomes and fitted CVD-risk curves

Validation outcomes remain 0 or 1; the fitted model estimates their conditional mean.

Curve

set T=0set T=1true DGP curvebands = 95% delta-method pointwise CI using the selected covariance

Empirical points are observed age-bin risks within the actual smoking groups. Fitted curves can be reference-profile curves or averages over a common covariate distribution.

4. Coefficients, standard errors, and conditional odds ratios

Click a row to update the coefficient-focused mathematical interpretation.

Term	β̂	SE	z	p	exp(β̂)	95% exp(β) CI

Interpretation: exp(β̂) is a conditional odds ratio only for a meaningful one-unit covariate contrast with the remaining model terms held fixed. For the intercept it is baseline odds; for Age² it is a curvature parameter; and for Smoker × age it is a ratio of odds ratios.

5. Outcome logistic model → standardized marginal risks

Every target-cohort subject is predicted twice, once under T=0 and once under T=1.

6. Treatment logistic model → propensity scores

A separate logistic regression estimates e(X)=P(T=1|X) and reveals overlap and extreme weights.

High treatment-prediction accuracy is not the goal. For causal weighting, inspect overlap, balance, and the concentration of inverse-probability weights.

Lecture NHANES reference results and exports

One-predictor age model

Complete-case classroom sample: n=6,299; 646 CVD events. The lecture fit was β̂₀=−6.172 and β̂_age=0.07247 per year.

Quantity	Lecture value
OR per year	1.075
OR per 10 years	2.064
Residual deviance	3605.6
AIC	3609.6

Multivariable outcome model and standardization

Term / estimand	Lecture value
Smoking conditional OR	1.539
BMI OR per unit	1.031
Income-ratio OR per unit	0.833
Standardized risk T=0	8.53%
Standardized risk T=1	11.98%
Marginal RD	3.45 pp
Marginal RR	1.405