I’ve been thinking a lot about model explainability lately because in my work with machine learning, I’ve seen too many brilliant models fail in production simply because stakeholders couldn’t understand how they reached their decisions. Whether it’s a bank rejecting a loan application or a healthcare system suggesting a treatment, people need to trust the AI they’re using. That’s why I want to share my practical experience with SHAP and LIME—two tools that have transformed how I build and deploy models.

Have you ever trained a model that performed perfectly on test data but left you scratching your head when asked to explain its predictions? I certainly have, and that’s where explainability frameworks come in. They help us peer inside the “black box” of complex models, revealing which features drive predictions and why specific decisions are made. This isn’t just academic—it’s becoming essential for regulatory compliance and ethical AI development.

Let’s start by setting up our environment. I prefer using a virtual environment to keep dependencies clean.

pip install shap lime scikit-learn pandas numpy matplotlib seaborn xgboost

Now, let’s import the necessary libraries. I always include these in my explainability projects.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import shap
import lime
from lime.lime_tabular import LimeTabularExplainer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
np.random.seed(42)

I’ll create a sample dataset for customer churn prediction, something I’ve worked with extensively. This helps illustrate the concepts without needing real sensitive data.

def create_sample_data():
    n_samples = 1000
    data = {
        'age': np.random.normal(45, 15, n_samples),
        'income': np.random.normal(50000, 20000, n_samples),
        'account_balance': np.random.exponential(1000, n_samples),
        'transaction_count': np.random.poisson(30, n_samples),
        'customer_tenure': np.random.uniform(0, 10, n_samples)
    }
    df = pd.DataFrame(data)
    df['churn_risk'] = (0.3 * (df['age'] > 60) + 
                        0.4 * (df['income'] < 30000) + 
                        0.3 * (df['account_balance'] < 500) + 
                        np.random.normal(0, 0.1, n_samples))
    df['churn'] = (df['churn_risk'] > 0.5).astype(int)
    return df

customer_data = create_sample_data()
X = customer_data.drop(['churn', 'churn_risk'], axis=1)
y = customer_data['churn']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)
model = RandomForestClassifier(n_estimators=100)
model.fit(X_train, y_train)

What makes SHAP so powerful is its foundation in game theory—it fairly distributes the “credit” for a prediction among all features. Here’s how I typically use it for global feature importance.

explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)
shap.summary_plot(shap_values, X_test)

This plot shows which features most influence your model’s output. But what if you need to explain a single prediction? That’s where SHAP’s force plots come in handy.

shap.force_plot(explainer.expected_value[1], shap_values[1][0,:], X_test.iloc[0,:])

Now, let’s talk about LIME. While SHAP provides consistent explanations, LIME focuses on creating local approximations. I find it particularly useful for text and image models, but it works well for tabular data too.

explainer_lime = LimeTabularExplainer(X_train.values, 
                                      feature_names=X_train.columns, 
                                      class_names=['No Churn', 'Churn'], 
                                      mode='classification')
exp = explainer_lime.explain_instance(X_test.values[0], model.predict_proba, num_features=5)
exp.show_in_notebook(show_table=True)

Have you noticed how LIME can sometimes give different explanations for similar instances? That’s because it builds a simple model around each prediction, which makes it fast but occasionally inconsistent. SHAP, on the other hand, guarantees consistency but can be computationally expensive for large datasets.

When working with complex models like neural networks, I often use SHAP’s KernelExplainer as a fallback.

background = shap.kmeans(X_train, 10)
explainer_kernel = shap.KernelExplainer(model.predict_proba, background)
shap_values_kernel = explainer_kernel.shap_values(X_test.iloc[0:10])

One common mistake I’ve made is not sampling enough background data for SHAP, which leads to unstable results. Always use a representative sample—I typically use 100-1000 instances depending on dataset size.

Another pitfall: forgetting that explainability tools can themselves introduce bias. If your background data isn’t diverse, your explanations might miss important patterns. I always validate explanations across different demographic segments when working with human-facing applications.

In healthcare projects, I’ve used SHAP to explain why a model flagged certain patients as high-risk. This transparency helped doctors trust the system enough to use it in clinical decisions. Similarly, in finance, LIME explanations have helped compliance teams understand credit scoring models.

What questions should you ask when choosing between SHAP and LIME? Consider your need for consistency versus speed, and whether you need global or local explanations. For most projects, I start with SHAP for its theoretical grounding, then use LIME for quick iterations.

Remember that no explainability method is perfect—they all make approximations. The goal is to provide enough insight for humans to make informed decisions about AI systems.

I’d love to hear about your experiences with model explainability. What challenges have you faced? Share your thoughts in the comments below, and if this guide helped you, please like and share it with others who might benefit from clearer model explanations.