Churn Watch
From churn reporting to churn prediction · IBM telco sample, 7,043 customers ·
every number below comes from the semantic layer or the evaluation in
results/metrics.jsonCustomers7,043snapshot
Churn rate26.5%historical: who already left
Monthly revenue lost$139,131historical: from customers who left
Monthly revenue at risk$68,294predicted: from customers still here
Avg. 3-year customer value$1,501survival-based, discounted
With a budget to contact 10% of customers (704), ranking by the model's churn
probability times each customer's value earned $35,831 more in the backtest than ranking by
historical segment churn rates (95% CI $21,065 to $50,672), assuming a $50 offer
that keeps 30% of would-be churners.
Where churn happens
Prediction versus the report
| Ranking | AUC | PR-AUC | Brier | Top-decile lift |
|---|---|---|---|---|
| Segment report (historical rates) | 0.831 | 0.605 | 0.140 | 2.71x |
| Logistic regression | 0.845 | 0.655 | 0.136 | 2.84x |
| Gradient-boosted trees | 0.846 | 0.658 | 0.135 | 2.86x |
Retention budget backtest
Contact first: highest expected loss among active customers
| Customer | Contract | Tenure (mo) | Monthly | Churn probability | 3-year value | Expected loss |
|---|---|---|---|---|---|---|
| 7145-FEJWU | Month-to-month | 12 | $105 | 72% | $2,455 | $1,776 |
| 8161-QYMTT | Month-to-month | 7 | $94 | 82% | $2,168 | $1,773 |
| 5150-ITWWB | Month-to-month | 3 | $95 | 82% | $2,133 | $1,743 |
| 3320-VEOYC | Month-to-month | 14 | $96 | 78% | $2,231 | $1,743 |
| 9094-AZPHK | Month-to-month | 15 | $100 | 74% | $2,350 | $1,738 |
| 0365-BZUWY | Month-to-month | 17 | $103 | 72% | $2,407 | $1,732 |
| 7225-CBZPL | Month-to-month | 17 | $95 | 78% | $2,225 | $1,727 |
| 4298-OYIFC | Month-to-month | 15 | $103 | 71% | $2,428 | $1,725 |
| 3096-GKWEB | Month-to-month | 18 | $95 | 77% | $2,220 | $1,716 |
| 3292-PBZEJ | Month-to-month | 11 | $111 | 66% | $2,594 | $1,712 |
| 6734-GMPVK | Month-to-month | 5 | $105 | 70% | $2,417 | $1,700 |
| 3452-GWUIN | Month-to-month | 19 | $95 | 77% | $2,212 | $1,696 |
| 5956-YHHRX | Month-to-month | 21 | $104 | 70% | $2,407 | $1,684 |
| 7398-SKNQZ | Month-to-month | 3 | $95 | 79% | $2,134 | $1,684 |
| 7392-YYPYJ | Month-to-month | 22 | $101 | 72% | $2,323 | $1,667 |
| 5935-FCCNB | Month-to-month | 17 | $94 | 75% | $2,211 | $1,652 |
| 9919-KNPOO | Month-to-month | 21 | $94 | 76% | $2,176 | $1,645 |
| 8313-NDOIA | Month-to-month | 24 | $104 | 68% | $2,388 | $1,632 |
| 9290-SHCMB | Month-to-month | 13 | $97 | 72% | $2,267 | $1,629 |
| 6350-XFYGW | Month-to-month | 4 | $95 | 75% | $2,158 | $1,627 |
| 0916-QOFDP | Month-to-month | 14 | $96 | 73% | $2,236 | $1,624 |
| 7668-XCFYV | Month-to-month | 17 | $93 | 75% | $2,172 | $1,624 |
| 6215-NQCPY | Month-to-month | 15 | $100 | 69% | $2,340 | $1,620 |
| 4927-WWOOZ | Month-to-month | 2 | $91 | 80% | $2,027 | $1,617 |
| 8087-LGYHQ | Month-to-month | 9 | $94 | 74% | $2,180 | $1,616 |
What drives the model
| Feature | Odds ratio |
|---|---|
| tenure_months | 0.24 |
| contract_Two year | 0.46 |
| total_charges | 1.98 |
| contract_Month-to-month | 1.83 |
| phone_service_True | 0.57 |
| internet_service_No | 0.58 |
| monthly_charges | 1.47 |
| internet_service_Fiber optic | 1.45 |
Numeric features are standardized, so their odds ratios are per standard deviation.
Fairness check
| Attribute | Group | Customers | AUC | Avg. predicted | Observed |
|---|---|---|---|---|---|
| gender | Female | 3,488 | 0.848 | 26.8% | 26.9% |
| gender | Male | 3,555 | 0.842 | 26.3% | 26.2% |
| senior citizen | False | 5,901 | 0.846 | 24.1% | 23.6% |
| senior citizen | True | 1,142 | 0.802 | 39.1% | 41.7% |
Gender and senior status are not model inputs. The model works equally well by gender, but it ranks senior customers less accurately and under-predicts their churn, so seniors would be slightly less likely to get an offer. That gap should be monitored, or corrected with a group-level calibration check before any campaign.