How do I explain that decision? A new approach combining counterfactuals and sensitivity analyses

Machine learning models increasingly integrate human decision-making. However, their black-box nature necessitates methods to explain their decisions. While popular explainable methods highlight influential features for predictions, they rarely guide agents on how to improve their performance or change their status. We propose a method that combines counterfactual analysis and Shapley effects to provide insights into improvement directions, while avoiding model predictions at impossible points. We carefully analyze the construction of appropriate Shapley explanations considering the recent criticism by Marques-Silva and Huang (2024). Numerical experiments indicate that the method offers valuable individual-level recommendations, highlighting the actionable features. At a system level, the method provides insights for risk management and allows investigation of whether critical features exhibit discriminatory traits or bias. To address the impossible points encountered in the calculation, we define the Shapley value only requires model evaluation on the feasible points so that the impossible move confounds in the interactions.