Churn Prediction: From Logistic Regression to Foundation Models

Introduction

Customer churn - the rate at which customers stop doing business with a company - remains one of the most critical metrics in subscription economies, SaaS, telecom, and financial services. A 5% reduction in churn can increase profits by 25-95% depending on the industry.

Yet the approaches to predicting churn have evolved dramatically. What started with simple logistic regression has progressed through ensemble methods to today's time-series foundation models like Google's TimesFM 2.5 and Amazon's Chronos 2.

This article provides a technical comparison across three paradigms:

• Statistical Methods: Logistic regression, survival analysis
• Machine Learning: XGBoost, LightGBM, Random Forest
• Foundation Models: TimesFM 2.5, Chronos 2

We'll examine architectures, performance characteristics, and when to use each approach.

The Churn Prediction Problem: A Technical Framing

Churn prediction can be formulated in multiple ways:

The choice of formulation affects which methods are applicable and how performance should be measured.

Statistical Methods: The Foundation

Logistic Regression

Logistic regression remains the baseline for churn prediction. Its interpretability makes it valuable for regulatory environments and stakeholder communication.

Mathematical Formulation:

P(churn = 1 | X) = 1 / (1 + e^-(β₀ + β₁x₁ + ... + βₙxₙ))

Strengths:

• Fully interpretable coefficients (odds ratios)
• No hyperparameter tuning required
• Works with small datasets (n < 1000)
• Regulatory-friendly (GDPR, fair lending)

Limitations:

• Assumes linear relationship between log-odds and features
• Cannot capture complex feature interactions
• Requires manual feature engineering

from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler

# Feature engineering for churn
features = [
    'days_since_last_login',
    'avg_session_duration_30d',
    'support_tickets_90d',
    'payment_failures_count',
    'contract_months_remaining'
]

model = LogisticRegression(
    penalty='l2',
    C=1.0,
    class_weight='balanced'  # Handle churn imbalance
)
model.fit(X_train[features], y_train)

# Interpretable coefficients
for feat, coef in zip(features, model.coef_[0]):
    odds_ratio = np.exp(coef)
    print(f"{feat}: OR = {odds_ratio:.2f}")

Survival Analysis: Time-to-Event Modeling

When the question shifts from "will they churn?" to "when will they churn?", survival analysis becomes essential.

Cox Proportional Hazards Model:

The hazard function represents the instantaneous risk of churning at time t, given survival until t:

h(t|X) = h₀(t) · e^(β₁x₁ + ... + βₙxₙ)

Key Advantages:

• Handles right-censoring (customers still active at observation end)
• Produces survival curves showing retention probability over time
• Enables hazard ratios for interpretable risk factors

from lifelines import CoxPHFitter

# Prepare survival data
survival_df = df[['tenure_days', 'churned', 'plan_type',
                   'usage_intensity', 'support_contacts']]

cph = CoxPHFitter(penalizer=0.1)
cph.fit(survival_df, duration_col='tenure_days', event_col='churned')

# Hazard ratios
cph.print_summary()

# Predict median survival time for new customers
median_survival = cph.predict_median(new_customer_features)

Machine Learning: Gradient Boosting Dominance

XGBoost / LightGBM

Gradient boosting methods dominate production churn systems due to their balance of performance, interpretability, and operational simplicity.

Why Gradient Boosting Excels at Churn:

1. Automatic feature interactions: Captures non-linear relationships without manual engineering
2. Handles mixed data types: Categorical and numerical features natively
3. Missing value robustness: Built-in handling of NULL values
4. Feature importance: SHAP values provide interpretability

import xgboost as xgb
from sklearn.model_selection import TimeSeriesSplit

# Churn-specific hyperparameters
params = {
    'objective': 'binary:logistic',
    'eval_metric': 'auc',
    'scale_pos_weight': (y_train == 0).sum() / (y_train == 1).sum(),
    'max_depth': 6,
    'learning_rate': 0.05,
    'subsample': 0.8,
    'colsample_bytree': 0.8,
    'early_stopping_rounds': 50
}

# Time-aware cross-validation (critical for churn)
tscv = TimeSeriesSplit(n_splits=5)
for train_idx, val_idx in tscv.split(X):
    model = xgb.XGBClassifier(**params)
    model.fit(
        X.iloc[train_idx], y.iloc[train_idx],
        eval_set=[(X.iloc[val_idx], y.iloc[val_idx])],
        verbose=False
    )

Feature Engineering for ML Churn Models

The success of ML models depends heavily on temporal feature engineering:

Foundation Models: The New Paradigm

Why Time Series Foundation Models for Churn?

Traditional approaches treat churn as a static classification problem. But customer behavior is inherently sequential - a trajectory of interactions over time.

Time-series foundation models are pre-trained on billions of time series across domains, learning universal patterns of:

• Trend detection
• Seasonality decomposition
• Anomaly identification
• Regime change detection

These capabilities transfer directly to churn prediction: detecting when a customer's engagement trajectory deviates from healthy patterns.

TimesFM 2.5 (Google)

TimesFM 2.5 is Google's latest time-series foundation model, released in December 2024. It's a 200M parameter decoder-only transformer pre-trained on 100B+ real-world time points.

Key Features of TimesFM 2.5:

• Zero-shot forecasting: No fine-tuning required
• Multi-horizon: Predicts 1-128 steps ahead simultaneously
• Frequency agnostic: Works across seconds to years
• Fine-tuning support: Can be adapted to domain-specific patterns

import torch
from timesfm import TimesFm

# Load pretrained model
model = TimesFm(
    context_len=512,
    horizon_len=30,  # Predict 30 days ahead
    input_patch_len=32,
    output_patch_len=128,
    num_layers=24,
    model_dims=1024
)
model.load_from_checkpoint('timesfm-2.5-200m')

# Prepare customer engagement time series
# Shape: (batch_size, context_length)
customer_series = torch.tensor([
    daily_logins[-512:],      # Last 512 days of login counts
])

# Zero-shot forecast
future_engagement = model.forecast(customer_series)

# Churn signal: engagement drops >50% from baseline
baseline = customer_series[:, -30:].mean(dim=1)
predicted_avg = future_engagement.mean(dim=1)
churn_risk = (predicted_avg / baseline) < 0.5

Chronos 2 (Amazon)

Chronos 2, released in late 2024, takes a different approach: it tokenizes continuous time series values into discrete tokens, treating forecasting as a language modeling problem.

Key Differentiators:

• Probabilistic outputs: Native uncertainty quantification
• Language model transfer: Leverages T5/Llama pre-training
• Robust to scale: Tokenization handles diverse value ranges
• Multi-series: Concurrent forecasting across customer cohorts

from chronos import ChronosPipeline

# Load Chronos 2 (various sizes: tiny, mini, small, base, large)
pipeline = ChronosPipeline.from_pretrained(
    "amazon/chronos-t5-large",
    device_map="cuda"
)

# Customer engagement series
context = torch.tensor([
    customer_metrics_df['daily_active_minutes'].values[-512:]
])

# Generate probabilistic forecasts
forecast = pipeline.predict(
    context,
    prediction_length=30,
    num_samples=100  # Monte Carlo samples for uncertainty
)

# Churn probability = P(future_engagement < threshold)
threshold = context.mean() * 0.3  # 70% drop = churn
churn_prob = (forecast < threshold).float().mean()

Head-to-Head Comparison

Performance Benchmarks

Based on published benchmarks and internal experiments on SaaS churn datasets:

When to Use Each Approach

Ensemble Strategy

Combining XGBoost (tabular features) with TimesFM (temporal patterns) often yields the best results:

from sklearn.linear_model import LogisticRegression

# Level 1: Base models
xgb_probs = xgb_model.predict_proba(X_tabular)[:, 1]
timesfm_probs = compute_churn_from_forecast(timesfm_model, X_temporal)

# Level 2: Meta-learner
meta_features = np.column_stack([xgb_probs, timesfm_probs])
meta_model = LogisticRegression()
meta_model.fit(meta_features, y_train)

# Final prediction
final_churn_prob = meta_model.predict_proba(meta_features)[:, 1]

Key Takeaways

• Logistic regression remains valuable for interpretability and regulatory compliance, but leaves performance on the table.

• Gradient boosting (XGBoost/LightGBM) offers the best balance of performance, interpretability, and operational simplicity for most production systems.

• Survival analysis is essential when predicting when churn occurs, not just if - critical for lifetime value optimization.

• TimesFM 2.5 and Chronos 2 represent a paradigm shift: treating customer behavior as time series enables zero-shot prediction and captures sequential patterns that tabular models miss.

• Fine-tuned foundation models can outperform XGBoost but require significant temporal data and GPU infrastructure.

• Ensemble approaches combining tabular ML with temporal foundation models often achieve the best results.

• The right choice depends on your constraints: data availability, interpretability requirements, infrastructure, and whether you need point predictions or probabilistic forecasts.

The evolution from logistic regression to foundation models mirrors the broader trajectory of ML: from hand-crafted features to learned representations, from task-specific models to transfer learning. For churn prediction, we're now at an inflection point where pre-trained temporal reasoning can be applied to customer behavior with minimal adaptation.

The question is no longer whether transformer-based models can predict churn - they demonstrably can. The question is whether your infrastructure, data, and use case justify the added complexity over well-tuned gradient boosting.

For most organizations, the answer is a hybrid approach: XGBoost for reliable, interpretable production serving, with foundation models for specialized high-value cohorts where the additional signal justifies the cost.

Ready to implement advanced churn prediction? Reach out to explore how ML-powered customer analytics can reduce churn and maximize lifetime value.