Optimized Fraud Detection in FinTech Transactions Using Genetic Algorithm and Random Forest Hybridization

Abstract

Financial fraud presents significant challenges to FinTech platforms due to the escalating volume and complexity of digital transactions. Current detection methods frequently encounter difficulties with imbalanced datasets, high dimensional characteristics, and the evolution of fraud trends. The suggested study presents a hybrid model combining Genetic Algorithm and Random Forest (GA–RF) to enhance feature selection and hyperparameter optimisation for effective fraud detection. The Genetic Algorithm determines the most informative feature subsets and best parameters for the Random Forest, while the Random Forest classifier utilises ensemble learning for accurate predictions. Experiments were performed on publicly accessible Kaggle datasets, including IEEE-CIS Fraud Detection, PaySim, and BankSim, which involve millions of simulated and actual transactions. The suggested hybrid method surpassed baseline models—Logistic Regression, SVM, Decision Tree, and XGBoost—attaining an accuracy of 99.3%, precision of 98.7%, recall of 97.9%, F1-score of 98.3%, and AUC-ROC of 0.996. These findings underscore the framework's capacity to address class imbalance, minimise false positives, and enhance model interpretability. The research illustrates that GA–RF hybridisation delivers a scalable, high-performance, and practical approach for proactive fraud detection, yielding considerable improvement in accuracy, sensitivity, and operational applicability compared to traditional machine learning techniques.

https://doi.org/10.65470/james.4