About the package¶
- Package website: https://norskregnesentral.github.io/shapr/
- Includes package information, introductions, examples etc.
- Source code: https://github.com/NorskRegnesentral/shapr
- Package paper (Sellereite & Jullum, JOSS, 2020) https://joss.theoj.org/papers/10.21105/joss.02027
Installation¶
Via GitHub¶
- Standard version:
In [1]:
#remotes::install_github("NorskRegnesentral/shapr") # Remove initial # to run
- Ctree version
In [2]:
#remotes::install_github("NorskRegnesentral/shapr",ref = "ctree") # Remove initial # to run
From CRAN¶
- Release version (available within a week or two)
In [3]:
#install.packages("shapr") # Remove initial # to run
How to use the package¶
- Prepare the explanation with the shapr function
- Input
- Training data
- Model to explain
- Additional parameters related to the KernelSHAP approximation
- Input
- Compute the Shapley values with the explain function
- Input
- The output from shapr
- Data whose predictions ought to be explained (test data)
- String specifying the approach to estimate the conditional expectations
- The value to use as reference $\phi_0$ (typically the mean of the responses or predictions on the training data)
- Additional specifications for the conditional expectation approach
- Input
- Either
- Print the table of Shapley values
- Plot the Shapley values with plot.shapr
Example explaining a risk model for loan customers¶
- Data set: Home Equity Line of Credit (HELOC) applications made by homeowners (retrived from https://community.fico.com/s/explainable-machine-learning-challenge)
Response: Binary
- 1 = Bad (>90 days late with a payment)
- 0 = Good (otherwise)
Examples:
- First explain a model with only numerical featuresthen add categorical features
- Then explain a model with both numerical and categorical features
Model: XGBoost model (Tree-based gradient boosting machine) + additional logistic regression model for example 2 for illustation
In [4]:
#### Package usage ####
library(shapr)
library(xgboost)
library(data.table)
library(ggplot2)
data <- data.table::fread(file = "FICO_HELOC_dataset_v1.csv")
Prepare modelling data¶
In [5]:
# Define response
data[,y:=(RiskPerformance=="Bad")*1]
data[,RiskPerformance:=NULL]
# NA's are coded as Negative numbers -- transform them
data[data<0]=NA
# Re-code two factor variables
data[,MaxDelq2PublicRecLast12M:=as.factor(MaxDelq2PublicRecLast12M)]
data[,MaxDelqEver:=as.factor(MaxDelqEver)]
# Define response and variables
y_var <- "y"
x_var_numeric = c("ExternalRiskEstimate",
"PercentTradesNeverDelq",
"NetFractionRevolvingBurden",
"NumTrades60Ever2DerogPubRec",
"MSinceMostRecentInqexcl7days")
x_var_factor = c("MaxDelq2PublicRecLast12M", "MaxDelqEver")
x_var_mixed = c(x_var_numeric,x_var_factor)
xy_var = c(x_var_mixed,y_var)
xy_data = data[,..xy_var]
xy_data = xy_data[complete.cases(xy_data)] # Remove observations with one or more NA in our features
In [6]:
# Split data in training and test set
set.seed(123)
test_ind = sort(sample(1:nrow(xy_data),5))
train_ind = (1:nrow(xy_data))[-test_ind]
x_train <- xy_data[train_ind,..x_var_numeric]
y_train <- unlist(xy_data[train_ind,..y_var])
x_test <- xy_data[test_ind,..x_var_numeric]
Fit XGBoost model to numerical features¶
In [7]:
# Fitting a basic xgboost model to the training data
params <- list(eta = 0.3,
max_depth = 4,
objective= 'binary:logistic',
eval_metric = "auc",
tree_method="hist")
model <- xgboost::xgboost(
data = as.matrix(x_train),
label = y_train,
nround = 20,
params = params,
verbose = T)
In [8]:
# Prepare the data/model for explanation
explainer <- shapr(x = x_train,
model = model)
# Define the reference prediction for explanation
p <- mean(y_train)
In [9]:
explanation_empirical <- explain(
x = x_test,
approach = "empirical",
explainer = explainer,
prediction_zero = p
)
Print and plot the Shapley values¶
In [10]:
# Print
explanation_empirical$dt
# Plot including phi_0
#plot(explanation_empirical)
# Plot excluding phi_0
#plot(explanation_empirical,plot_phi0 = F)
| With phi0 | Without phi0 |
|---|---|
 |
 |
Same empirical explanation method, but increasing the bandwidth¶
In [11]:
## Incrasing the bandwidth parameter
explanation_empirical_02 <- explain(
x = x_test,
approach = "empirical",
type = "fixed_sigma",
fixed_sigma_vec = 0.2,
explainer = explainer,
prediction_zero = p
)
# Print
explanation_empirical_02$dt
Empirical approach assuming independence¶
In [12]:
# Assuming independence
explanation_empirical_indep <- explain(
x = x_test,
approach = "empirical",
type = "independence",
explainer = explainer,
prediction_zero = p
)
# Print
explanation_empirical_indep$dt
Gaussian approach¶
In [13]:
# Using the Gaussian approach
explanation_gaussian <- explain(
x = x_test,
approach = "gaussian",
explainer = explainer,
prediction_zero = p
)
# Print
#print(explanation_gaussian$dt)
explanation_gaussian$dt
Ctree approach¶
In [14]:
# Using the ctree approach
explanation_ctree <- explain(
x = x_test,
approach = "ctree",
explainer = explainer,
prediction_zero = p
)
# Print
explanation_ctree$dt
In [15]:
x_train_mixed <- xy_data[train_ind,..x_var_mixed]
x_test_mixed <- xy_data[test_ind,..x_var_mixed]
For simplicty we first do this for a logistic regression model¶
In [16]:
model_glm_mixed = glm(y~.,data=cbind(x_train_mixed,y=as.factor(y_train)),family="binomial")
explainer_glm_cat <- shapr(x = x_train_mixed,
model = model_glm_mixed)
explanation_glm_cat_ctree <- explain(
x = x_test_mixed,
approach = "ctree",
explainer = explainer_glm_cat,
prediction_zero = p)
explanation_glm_cat_ctree$dt
Mixed data with XGBoost model¶
- Since XGBoost does not handle categorical data directly, we need to transform the data to numerical using one-hot encoding
- We want to explain the original features and not the one-hot encoded features
- We therefore need to pass the tranformation function to our explainer to allow prediction of the XGBoost model using the original data
- We have create the special functionality to handle this:
- Call our function shapr::make_dummies to create the transformation function
- Get transformed data using our function shapr::apply_dummies
In [17]:
# Making the transformation function
dummylist <- make_dummies(data = rbind(x_train_mixed, x_test_mixed))
# Applying the transformation function
x_train_mixed_dummy <- apply_dummies(obj = dummylist, newdata = x_train_mixed)
# Fitting the XGBoost model
model_mixed <- xgboost(
data = as.matrix(x_train_mixed_dummy),
label = y_train,
nround = 20,
params = params,
verbose = F)
# Include the transformation function to the model object
model_mixed$dummylist <- dummylist
Explain the model just like before (ctree only!)¶
In [18]:
explainer_mixed <- shapr(x_train_mixed,
model_mixed)
explanation_ctree_mixed <- explain(
x_test_mixed,
approach = "ctree",
explainer = explainer_mixed,
prediction_zero = p
)
explanation_ctree_mixed$dt