Hyperparameter Optimization of Artificial Neural Network in Customer Churn Prediction using Genetic Algorithm

customers is considered to be worthy competitive advantage since it improves cost allocation
in customer retention programs, retaining future revenue and profits. In addition, it has several
positive indirect impacts such as increasing customer’s loyalty. Therefore, the focus of the article
is on building highly reliable and robust classification model, which deals with such a task.
Methodology/methods: The analysis is carried out on labelled ecommerce retail dataset
describing 10 000 most valuable customers with the highest CLV (Customer Lifetime Value).
To obtain the best performing ANN (Artificial Neural Network) classification model, proposed
hyperparameter search space is explored with genetic algorithm to find suitable parameter
settings. ANN classification performance is measured with regard to prediction ability, which
is understood as point estimate of AUC (Area Under Curve) mean on 4fold cross-validation
set. Explored part of hyperparameter search space is analyzed with conditional inference tree
structure addressing underlying fundamental context of given optimization which results in
identification of critical factors leading to well performing ANN classification model.
Scientific aim: To present and execute experimental design for performance evaluation and
hyperparameter optimization of classification models, which are used for customer churn prediction.
Findings: It is concluded and statistically proven that in experimental context described,
regularization parameter as well as training function have significant influence on classifiers
AUC performance contrasting other properties of ANN. More specifically, well performing
ANN classification models have regularization parameter set to 0, adaptation function set to
trainlm or trainscg and more than 100 training epochs. Global optimum is identified for solution
with regularization parameter set to 0, trainlm adaptation function, 350 training epochs and
7-4-2 architecture.
Conclusions: Results imply that placing hyperparameter optimization to ANN classification model
leads to improved customer churn prediction ability. The article describes design and execution of
machine learning pipeline, hyperparameter optimization and original meta-analysis of the results
with conditional inference tree structure, which are considered beneficial for further research.