Automated Detection of Pavement Fatigue Cracks with Deep Learning

Alipour, Shaghayegh; Khodadadi, Mojtaba; Amani, Mohammad Javad; Khodaii, Ali; Hesami, Saeid

doi:10.61882/NMCE.2511.1110

Automated Detection of Pavement Fatigue Cracks with Deep Learning

Document Type : Research

Authors

Shaghayegh Alipour ¹

Mojtaba Khodadadi ²

Mohammad Javad Amani ³

Ali Khodaii ³

Saeid Hesami ¹

¹ Department of Civil Engineering, Babol Noshirvani University of Technology, Mazandaran, Iran

² Faculty of Civil Engineering at K. N. Toosi University of Technology, Tehran, Iran

³ Department of Civil and Environmental Engineering, Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran

10.61882/NMCE.2511.1110

Abstract

This study uses advanced imaging techniques and deep learning algorithms to assess fatigue cracks through cyclic loading on asphalt specimens. Faster Region-Based Convolutional Neural Network (Faster R-CNN) and the You Only Look Once (YOLO) models were compared to detect fatigue cracks in Ground-Penetrating Radar (GPR) and Computed Tomography (CT) scan images and to detect concealed cracks in GPR field data. Crack detection was improved using transfer learning with pre-trained weights from the COCO dataset. Using the piecewise function model, the accumulative horizontal strain was accurately estimated. Based on the statistical analysis, the model's accuracy was verified, with no significant differences between experimental and predicted results. Moreover, a piecewise function applied to CT scan data resulted in a better understanding of fatigue behavior. The crack classification was improved after retraining pre-trained deep convolutional neural networks (PDCNNs). The YOLO models outperformed Faster-RCNN in terms of average precision. Models YOLOv7, YOLOv5s, and YOLOv8 performed well on the GPR dataset, while YOLOv5s, YOLOv5m, and YOLOv8 were the most effective models on the CT dataset.

Keywords

Fatigue Cracks

Ground-Penetrating Radar

CT-Scan

Asphalt Pavement

YOLO

Transfer Learning

Faster R-CNN

Object Detection

Subjects

Transportation Engineering

[1] H. Liu, X. Yang, L. Jiang, S. Lv, T. Huang, Y. Yang, Fatigue-creep damage interaction model of asphalt mixture under the semi-sine cycle loading, Constr. Build. Mater. 251 (2020) 119070. https://doi.org/10.1016/j.conbuildmat.2020.119070.

[2] Z. Zhang, S. Shen, B. Shi, H. Wang, Characterization of the fatigue behavior of asphalt mixture under full support using a Wheel-tracking Device, Constr. Build. Mater. 277 (2021) 122326. https://doi.org/10.1016/j.conbuildmat.2021.122326.

[3] A. Mahpour, S. Alipour, M. Khodadadi, A. Khodaii, J. Absi, Leaching and mechanical performance of rubberized warm mix asphalt modified through the chemical treatment of hazardous waste materials, Constr. Build. Mater. 366 (2023) 130184. https://doi.org/10.1016/j.conbuildmat.2022.130184.

[4] H. Qin, D. Zhang, Y. Tang, Y. Wang, Automatic recognition of tunnel lining elements from GPR images using deep convolutional networks with data augmentation, Autom. Constr. 130 (2021) 103830. https://doi.org/10.1016/j.autcon.2021.103830.

[5] J. Zhang, C. Zhang, Y. Lu, T. Zheng, Z. Dong, Y. Tian, Y. Jia, In-situ recognition of moisture damage in bridge deck asphalt pavement with time-frequency features of GPR signal, Constr. Build. Mater. 244 (2020) 118295. https://doi.org/10.1016/j.conbuildmat.2020.118295.

[6] Z. Tong, J. Gao, D. Yuan, Advances of deep learning applications in ground-penetrating radar: A survey, Constr. Build. Mater. 258 (2020) 120371. https://doi.org/10.1016/j.conbuildmat.2020.120371.

[7] J. Zhang, X. Yang, W. Li, S. Zhang, Y. Jia, Automatic detection of moisture damages in asphalt pavements from GPR data with deep CNN and IRS method, Autom. Constr. 113 (2020) 103119. https://doi.org/10.1016/j.autcon.2020.103119.

[8] Z. Liu, W. Wu, X. Gu, S. Li, L. Wang, T. Zhang, Application of combining yolo models and 3d gpr images in road detection and maintenance, Remote Sens. (Basel). 13 (2021) 1–18. https://doi.org/10.3390/rs13061081.

[9] M.G. Zamani, M.R. Nikoo, F. Niknazar, G. Al-Rawas, M. Al-Wardy, A.H. Gandomi, A multi-model data fusion methodology for reservoir water quality based on machine learning algorithms and bayesian maximum entropy, J. Clean. Prod. 416 (2023) 137885.

[10] W. Lei, F. Hou, J. Xi, Q. Tan, M. Xu, X. Jiang, G. Liu, Q. Gu, Automatic hyperbola detection and fitting in GPR B-scan image, Autom. Constr. 106 (2019) 102839. https://doi.org/10.1016/j.autcon.2019.102839.

[11] M. Rasol, J.C. Pais, V. Pérez-Gracia, M. Solla, F.M. Fernandes, S. Fontul, D. Ayala-Cabrera, F. Schmidt, H. Assadollahi, GPR monitoring for road transport infrastructure: A systematic review and machine learning insights, Constr. Build. Mater. 324 (2022). https://doi.org/10.1016/j.conbuildmat.2022.126686.

[12] F.M. Fernandes, J.C. Pais, Laboratory observation of cracks in road pavements with GPR, Constr. Build. Mater. 154 (2017) 1130–1138. https://doi.org/10.1016/j.conbuildmat.2017.08.022.

[13] M.A. Rasol, V. Pérez-Gracia, M. Solla, J.C. Pais, F.M. Fernandes, C. Santos, An experimental and numerical approach to combine Ground Penetrating Radar and computational modeling for the identification of early cracking in cement concrete pavements, NDT and E International 115 (2020). https://doi.org/10.1016/j.ndteint.2020.102293.

[14] M. Solla, S. Lagüela, H. González-Jorge, P. Arias, Approach to identify cracking in asphalt pavement using GPR and infrared thermographic methods: Preliminary findings, NDT and E International 62 (2014) 55–65. https://doi.org/10.1016/j.ndteint.2013.11.006.

[15] Z. Tong, J. Gao, H. Zhang, Recognition, location, measurement, and 3D reconstruction of concealed cracks using convolutional neural networks, Constr. Build. Mater. 146 (2017) 775–787. https://doi.org/10.1016/j.conbuildmat.2017.04.097.

[16] Z. Tong, D. Yuan, J. Gao, Y. Wei, H. Dou, Pavement-distress detection using ground-penetrating radar and network in networks, Constr. Build. Mater. 233 (2020) 117352. https://doi.org/10.1016/j.conbuildmat.2019.117352.

[17] J. Gao, D. Yuan, Z. Tong, J. Yang, D. Yu, Autonomous pavement distress detection using ground penetrating radar and region-based deep learning, Measurement (Lond). 164 (2020). https://doi.org/10.1016/j.measurement.2020.108077.

[18] Z. Liu, J.K.W. Yeoh, X. Gu, Q. Dong, Y. Chen, W. Wu, L. Wang, D. Wang, Automatic pixel-level detection of vertical cracks in asphalt pavement based on GPR investigation and improved mask R-CNN, Autom. Constr. 146 (2023) 104689. https://doi.org/10.1016/j.autcon.2022.104689.

[19] S. Li, X. Gu, X. Xu, D. Xu, T. Zhang, Z. Liu, Q. Dong, Detection of concealed cracks from ground penetrating radar images based on deep learning algorithm, Constr. Build. Mater. 273 (2021) 121949. https://doi.org/10.1016/j.conbuildmat.2020.121949.

[20] Z. Liu, X. Gu, J. Chen, D. Wang, Y. Chen, L. Wang, Automatic recognition of pavement cracks from combined GPR B-scan and C-scan images using multiscale feature fusion deep neural networks, Autom. Constr. 146 (2023) 104698. https://doi.org/10.1016/j.autcon.2022.104698.

[21] A.T. Papagiannakis, A. Abbas, E. Masad, Micromechanical analysis of viscoelastic properties of asphalt concretes, Transp. Res. Rec. 1789 (2002) 113–120.

[22] S. Saadeh, L. Tashman, E. Masad, W. Mogawer, Spatial and directional distribution of aggregates in asphalt mixes, J. Test. Eval. 30 (2002) 483–491.

[23] L. Banta, K. Cheng, J. Zaniewski, Estimation of limestone particle mass from 2D images, Powder Technol. 132 (2003) 184–189.

[24] L. Wang, H.S. Paul, T. Harman, J. D’Angelo, Characterization of aggregates and asphalt concrete using X-Ray computerized tomography-A state of the art report (with discussion), Journal of the Association of Asphalt Paving Technologists 73 (2004).

[25] R. Khan, A.C. Collop, G.D. Airey, A.N. Khan, Asphalt damage characterization from cyclic test and X-ray computed tomography, Proceedings of the Institution of Civil Engineers: Transport 166 (2013) 203–213. https://doi.org/10.1680/tran.11.00045.

[26] L. Gao, F. Ni, H. Luo, S. Charmot, Characterization of air voids in cold in-place recycling mixtures using X-ray computed tomography, Constr. Build. Mater. 84 (2015) 429–436. https://doi.org/10.1016/j.conbuildmat.2015.03.081.

[27] J. Hu, P. Liu, D. Wang, M. Oeser, Y. Tan, Investigation on fatigue damage of asphalt mixture with different air-voids using microstructural analysis, Constr. Build. Mater. 125 (2016) 936–945. https://doi.org/10.1016/j.conbuildmat.2016.08.138.

[28] D. Braz, R.C. Barroso, R.T. Lopes, L.M.G. Motta, Crack detection in asphaltic mixtures by computed tomography, NDT and E International 44 (2011) 195–201. https://doi.org/10.1016/j.ndteint.2010.11.005.

[29] Q. Li, H. Yang, X. Ma, F. Ni, Evaluation of microstructure and damage evolution for asphalt pavements in an advanced repeated load permanent deformation test using X-ray computed tomography, Road Materials and Pavement Design 18 (2017) 1135–1158. https://doi.org/10.1080/14680629.2016.1207555.

[30] Y. Li, W. Jiang, J. Shan, P. Li, R. Lu, B. Lou, Characteristics of void distribution and aggregate degradation of asphalt mixture specimens compacted using field and laboratory methods, Constr. Build. Mater. 270 (2021) 121488. https://doi.org/10.1016/j.conbuildmat.2020.121488.

[31] D. ASTM, 448-03. Standard Classification for Sizes of Aggregate for Road and Bridge Construction, in: American Society for Testing and Materials, 2003.

[32] M., Khodadadi, L., Moradi, B., Dabir, F.M. Nejad, and A., Khodaii, Reuse of drill cuttings in hot mix asphalt mixture: A study on the environmental and structure performance. Construction and Building Materials, 256, (2020) p.119453.

[33] M., Khodadadi, F. Moghadas Nejad, and A., Khodaii. Comparison of Rut Susceptibility Parameters in Modified Bitumen with PPA. AUT Journal of Civil Engineering, 1(2), (2017), pp.129-134.

[34] A., Mahpour, M., Khodadadi, M. Shahraki, and, F., Moghadas Nejad. Evaluation of moisture durability of modified asphalt mixture with nano-titanium dioxide using surface free energy method. Amirkabir Journal of Civil Engineering, 54(8), (2022) pp.2831-2850.

[35] M., Khodadadi, A. Azarhoosh, and A., Khodaii. Influence of polymeric coating the aggregate surface on moisture damage of hot mix asphalt. Periodica Polytechnica Civil Engineering, 65(2), 2021 pp.376-384.

[36] A. Khodaii, S. Fallah, F.M. Nejad, Effects of geosynthetics on reduction of reflection cracking in asphalt overlays, Geotextiles and Geomembranes 27 (2009) 1–8.

[37] F. Moghadas Nejad, A. Noory, S. Toolabi, S. Fallah, Effect of using geosynthetics on reflective crack prevention, International Journal of Pavement Engineering 16 (2015) 477–487.

[38] K. Sobhan, V. Tandon, Mitigating reflection cracking in asphalt overlays using geosynthetic reinforcements, Road Materials and Pavement Design 9 (2008) 367–387.

[39] D. Zamora-Barraza, M.A. Calzada-Pérez, D. Castro-Fresno, A. Vega-Zamanillo, Evaluation of anti-reflective cracking systems using geosynthetics in the interlayer zone, Geotextiles and Geomembranes 29 (2011) 130–136.

[40] M. Khodadadi, A. Khodaii, J. Absi, P. Hajikarimi, F.F. Tehrani, Multi-Length-Scale Investigation of the Fatigue Behavior of Bituminous Composites: Experimental Approach, Journal of Materials in Civil Engineering 37 (2025) 4025189. https://doi.org/10.1061/JMCEE7.MTENG-19135.

[41] M. Khodadadi, A. Khodaii, P. Hajikarimi, F. Fakhari Tehrani, J. Absi, Multi-Scale Numerical Viscoelastic Simulation of Fatigue Behavior of Asphalt Mixtures Modified with Polyphosphoric Acid, IOP Conf. Ser. Mater. Sci. Eng. 416 (2018) 12106. https://doi.org/10.1088/1757-899X/416/1/012106.

[42] F.M. Nejad, S. Asadi, S. Fallah, M. Vadood, Statistical-experimental study of geosynthetics performance on reflection cracking phenomenon, Geotextiles and Geomembranes 44 (2016) 178–187. https://doi.org/10.1016/j.geotexmem.2015.09.002.

[43] M. Khodadadi, A. Khodaii, J. Absi, P. Hajikarimi, F.F. Tehrani, Multi-Length-Scale Investigation of the Fatigue Behavior of Bituminous Composites: Numerical Approach, Journal of Materials in Civil Engineering 38 (2026) 4026011. https://doi.org/10.1061/JMCEE7.MTENG-21148.

[44] H. Harkat, A.E. Ruano, M.G. Ruano, S.D. Bennani, GPR target detection using a neural network classifier designed by a multi-objective genetic algorithm, Applied Soft Computing Journal 79 (2019) 310–325. https://doi.org/10.1016/j.asoc.2019.03.030.

[45] Z. Liu, X. Gu, H. Yang, L. Wang, Y. Chen, D. Wang, Novel YOLOv3 Model With Structure and Hyperparameter Optimization for Detection of Pavement Concealed Cracks in GPR Images, IEEE Transactions on Intelligent Transportation Systems 23 (2022) 22258–22268. https://doi.org/10.1109/TITS.2022.3174626.

[46] Z. Qiu, Z. Zhao, S. Chen, J. Zeng, Y. Huang, B. Xiang, Application of an Improved YOLOv5 Algorithm in Real-Time Detection of Foreign Objects by Ground Penetrating Radar, Remote Sens. (Basel). 14 (2022). https://doi.org/10.3390/rs14081895.

[47] J. Fan, J.H. Lee, I.S. Jung, Y.K. Lee, Improvement of Object Detection Based on Faster R-CNN and YOLO, 2021 36th International Technical Conference on Circuits/Systems, Computers and Communications, ITC-CSCC 2021 (2021) 3–6. https://doi.org/10.1109/ITC-CSCC52171.2021.9501480.

[48] D. Ma, H. Fang, N. Wang, B. Xue, J. Dong, F. Wang, A real-time crack detection algorithm for pavement based on CNN with multiple feature layers, Road Materials and Pavement Design 23 (2022) 2115–2131. https://doi.org/10.1080/14680629.2021.1925578.

[49] Q. Yang, W. Shi, J. Chen, W. Lin, Deep convolution neural network-based transfer learning method for civil infrastructure crack detection, Autom. Constr. 116 (2020) 103199. https://doi.org/10.1016/j.autcon.2020.103199.

[50] M.G. Zamani, M.R. Nikoo, D. Rastad, B. Nematollahi, A comparative study of data-driven models for runoff, sediment, and nitrate forecasting, J. Environ. Manage. 341 (2023) 118006.

[51] L. Ali, F. Alnajjar, H. Al Jassmi, M. Gochoo, W. Khan, M.A. Serhani, Performance evaluation of deep CNN-based crack detection and localization techniques for concrete structures, Sensors 21 (2021) 1–22. https://doi.org/10.3390/s21051688.

[52] M., Gholami, M., Khodadadi, P., Hajikarimi, and A., Khodaii. Investigating the effects of reducing the number of temperatures and frequencies on the development of master curves for viscoelastic properties of bituminous composite. Measurement, 230, (2024), p.114503.

[53] F. Zhou, T. Scullion, L. Sun, Verification and modeling of three-stage permanent deformation behavior of asphalt mixes, J. Transp. Eng. 130 (2004) 486–494.

[54] Z. Tong, J. Gao, H. Zhang, Recognition, location, measurement, and 3D reconstruction of concealed cracks using convolutional neural networks, Constr. Build. Mater. 146 (2017) 775–787. https://doi.org/10.1016/j.conbuildmat.2017.04.097.