Customer Churn Prediction — Harish Thiyagharajan Selvi

// model_results

Model Performance

Random Forest vs Logistic Regression

Threshold = 0.45 · Test set = 20% hold-out · SMOTE applied to training set only

Churn Rate by Contract Type

Month-to-month customers churn at 14× the rate of two-year contract holders

// feature_importance

What Drives Churn — Top 10 Features

Mean Decrease in Impurity · Amber bars = engineered features · Blue bars = original dataset features

Top signal

charge_x_tenure_risk

Monthly charge ÷ (tenure + 1) — captures billing shock for new customers. Our highest-value engineered feature.

Strongest raw feature

tenure

Months with the company. Customers in the first 12 months churn at 2–3× the average rate.

Biggest surprise

pays_by_electronic_check

Electronic check payers churn at 2× the rate of auto-pay customers. Manual payments = higher bill awareness.

// business_impact

Business Impact Estimate

Assumptions: $65 avg monthly revenue · 6 months retained per successful intervention · $25 cost per outreach call.

$491K estimated net benefit

On a 7,000-customer base, the model catches ~1,400 genuine churners. At $65/month × 6 months saved, the revenue retained is ~$546K. Intervention costs ~$55K. Net: ~$491K per cycle.

Threshold tuned to 0.45 (not 0.5)

Default 0.5 threshold misses too many real churners. At 0.45, we trade 4 pp of precision for 8 pp of recall — the right trade-off when a missed churner costs 6× more than an unnecessary call.

82% of retained = genuine — not guesses

84% recall means the retention team's outreach list is 84% genuine at-risk customers. The 16% miss rate still justifies the programme — no model is perfect at 30-day prediction.

// methodology

How It Was Built

1

Data quality assessmentTotalCharges stored as object dtype — whitespace strings masked nulls for tenure=0 customers. Fixed with pd.to_numeric(errors='coerce').fillna(0).

2

Feature engineering (22 features)Added tenure lifecycle groups, billing stress signals, service portfolio breadth, risk-combination flags, and charge×tenure interaction terms.

3

Train/test split (stratified)80/20 split with stratify=y to preserve 26.5% churn rate in both sets. SMOTE applied to training data only — never touched the test set.

4

SMOTE for class imbalanceSynthetic minority over-sampling balanced the training set 50/50. Combined with class_weight='balanced' in the RandomForestClassifier for double coverage.

5

GridSearchCV (ROC-AUC optimised)5-fold stratified CV over n_estimators, max_depth, min_samples_split, min_samples_leaf. Optimised for AUC not accuracy — accuracy is misleading with imbalance.

6

Threshold calibrationSwept 0.30–0.70 range on the precision-recall curve. Selected 0.45 as the point where recall ≥ 80% and net business benefit is maximised.

Challenges & Solutions

Challenge	Solution
Hidden nulls in TotalCharges	str.strip() → pd.to_numeric(errors='coerce') → fillna(0)
26.5% class imbalance	SMOTE on training set + class_weight='balanced' + threshold 0.45
Overfitting (100% train accuracy)	max_depth=12, min_samples_leaf=2 via GridSearchCV
"82% accuracy" means nothing	Report Recall + Precision + AUC + business impact instead
Stakeholders wanted "why"	Feature importance chart + plain-English business impact report

Why Random Forest over XGBoost?

XGBoost gave +0.008 AUC improvement at the cost of 4× hyperparameter complexity and harder-to-explain feature importance. For a retention team that needs to act on the model's output, interpretability wins.

// explore_more

Full Code & Notebook

The complete runnable Jupyter notebook, preprocessing scripts, feature engineering module, and model evaluation code are all on GitHub. The notebook auto-generates synthetic data so you can run it end-to-end without downloading anything.

View on GitHub ← All Projects

Customer ChurnPrediction Model