Machine Learning-Based Failure Detection in IoT-Enabled Engineering Systems: A Time-Aware and Imbalance-Sensitive Evaluation Using the MetroPT-3 Dataset

Publication Details

Publication Date

10/10/2026

Volume / Issue

Vol 1, Issue 5 (2026)

Article No.

042

Journal

British Journal of Contemporary Research

DOI

https://doi.org/10.67693/BJCR-9KU8493Z

Received

18 Sep 2026

Views

23

Downloads

14

Abstract

The integration of Internet of Things (IoT) technologies and machine learning (ML) has advanced intelligent monitoring and predictive maintenance of engineering systems. However, failure detection using industrial sensor data remains challenging because failure events are rare compared with normal operation, resulting in highly imbalanced datasets where accuracy alone may provide misleading conclusions. This study presents a time-aware and imbalance-sensitive evaluation of Decision Tree, Random Forest, and Extreme Gradient Boosting (XGBoost) classifiers for detecting failure-related conditions using the MetroPT-3 dataset. The dataset contains 1,516,948 multivariate time-series observations collected from a metro Air Production Unit (APU), including pressure, temperature, electrical, and digital control measurements. Documented failure periods were transformed into a binary classification problem, and a chronological 70:30 train-test split was applied to preserve temporal ordering and reduce information leakage. The test dataset contained 455,085 observations, including 1,622 failure cases (0.356%). Models were evaluated using accuracy, precision, recall, F1-score, balanced accuracy, MCC, PR-AUC, ROC-AUC, confusion matrices, and feature-importance analysis. Although all models achieved accuracy above 99%, failure-detection performance varied considerably. Decision Tree achieved 99.54% accuracy but detected only 0.86% of failures. Random Forest achieved 99.64% accuracy and zero recall at the default threshold but showed strong ranking capability with a ROC-AUC of 0.9973. XGBoost achieved the best minority-class performance with 60.74% precision, 39.40% recall, 47.79% F1-score, 0.6965 balanced accuracy, and 0.4877 MCC. Feature analysis identified DV_pressure, Oil_temperature, and TP2 as dominant predictors, with the first two contributing approximately 81.81% of XGBoost feature importance. The findings demonstrate that reliable failure detection requires temporal validation, imbalance-sensitive metrics, probability-based evaluation, and appropriate threshold selection rather than accuracy alone.

License

This article is published under the Creative Commons Attribution 4.0 International License . Free to read, share, and adapt with attribution.