Explainable Machine Learning for Real-Time Financial Fraud Detection and Risk-Based Transaction Monitoring in U.S. Banking and Digital Payment Systems

Authors

  • Masrul Hasan Master of Business Administration in Business Analytics; Trine University, United States Author

DOI:

https://doi.org/10.63125/4h9v1876

Keywords:

Explainable Machine Learning, Real-Time Fraud Detection, Risk-Based Transaction Monitoring, Ensemble Learning, SHAP, U.S. Digital Payments

Abstract

This study addresses real-time fraud detection in U.S. banking and digital payment systems, where transaction velocity, extreme class imbalance, adaptive adversaries, and supervisory expectations for transparent, risk-based decisions limit opaque or rule-only monitoring. It develops and evaluates an explainable ensemble machine-learning framework for real-time fraud scoring and risk-based transaction monitoring that integrates transaction and behavioral features, real-time processing, model explainability, and governance. A quantitative, cross-sectional, explanatory-predictive design was adopted. A structured questionnaire was administered to purposively sampled fraud-risk, compliance, model-development, and payment-operations professionals at U.S. banks, credit unions, payment processors, and fintech firms (n = 300 valid responses), and Random Forest, XGBoost, LightGBM, and a stacked ensemble were evaluated out of time on labeled transaction data (N = 2,400,000 illustrative transactions; fraud prevalence 0.62%). Analysis combined reliability and validity testing, Pearson correlation, multiple regression, cost-sensitive and ranking-based evaluation, probability calibration, latency testing, and SHAP-based explanation. Reliability was strong (Cronbach's α = .89–.93); the four constructs explained 68.1% of the variance in perceived monitoring effectiveness (TBFQ β = .36, RTPI β = .27, MEI β = .23, RGCM β = .19, all p ≤ .001); and the calibrated ensemble reached PR-AUC = 0.901 and ROC-AUC = 0.972 out of time, versus 0.887 and 0.968 for LightGBM, with recall of 0.76 at a 1% false-positive rate, a 99th-percentile latency of 118 ms, and tier fraud rates rising from 0.08% to 31.60%. Interpretable, calibrated ensemble scoring tied to explicit risk tiers can therefore strengthen alert prioritization, reduce customer friction, and support model-risk and compliance obligations as payment rails accelerate.

References

Downloads

Published

2025-10-10

How to Cite

Masrul Hasan. (2025). Explainable Machine Learning for Real-Time Financial Fraud Detection and Risk-Based Transaction Monitoring in U.S. Banking and Digital Payment Systems. American Journal of Data Science and Analytics, 6(10), 70-108. https://doi.org/10.63125/4h9v1876

Cited By: