Artificial Intelligence in Railroad Track Base Management: Applying Machine Perception and Predictive Analytics to Ballast, Subballast, and Subgrade Condition Assessment

Executive Summary

The condition of the railroad track base, the ballast, subballast, and subgrade layers beneath the rail and ties, governs the rate at which track geometry degrades, the drainage performance of the right of way, and ultimately the safety and cost profile of the network. Yet for most of the industry’s history, the substructure has been assessed indirectly: inspectors observe its surface symptoms, and regulators measure the geometry deviations it produces, while the layers themselves remain largely invisible to routine inspection.

Artificial intelligence is changing the economics and physics of that problem. Machine learning applied to ground penetrating radar (GPR) signals, deep learning computer vision applied to ballast imagery, and data fusion across geometry, radar, and satellite measurements now make it feasible to characterize substructure condition continuously, objectively, and at network scale. Federal Railroad Administration (FRA) research programs, university transportation centers, and the Class I railroads have each demonstrated components of this capability. What remains underdeveloped is the connective tissue: substructure-specific condition standards, validation protocols that establish equivalence with existing methods, shared labeled datasets, and a deliberate design for how human inspectors and automated systems divide the work.

This paper assesses the current state of AI in track base management, identifies five findings on capability maturity and adoption barriers, and offers recommendations for railroads, regulators, and researchers. The analysis draws on GAO oversight work, Congressional Research Service reporting, FRA regulatory and research publications, and peer-reviewed transportation geotechnics literature. Deeper treatments of individual technology threads, including GPR signal processing methods and substructure degradation modeling, are candidates for subsequent papers in this series.

1. Problem Statement

A ballasted track structure is a layered geotechnical system. The ballast layer holds the ties in position, distributes repeated wheel loads into the subgrade, and drains water away from the track. These functions degrade as ballast fouls: fine particles from ballast breakdown, subgrade intrusion, and dropped lading fill the void space between aggregate particles, reducing drainage capacity and resilience. Fouled, poorly draining ballast accelerates track deterioration, and certain fouling materials such as coal dust can act as lubricants that worsen permanent deformation of the granular layer. Severe fouling can lead to track instability and, in the worst case, derailment.

The inspection regime that monitors this system was built around what humans can see. Federal track safety standards, codified at 49 CFR Part 213, require periodic visual inspection on foot or from a hi-rail vehicle, a framework whose inspection frequencies date to the early 1970s. Visual inspection can identify surface symptoms of substructure distress, such as mud pumping at the ballast surface, but cannot measure layer thickness, internal fouling levels, or trapped moisture. The conventional direct method, excavating ballast samples for sieve analysis, is subjective in site selection, labor intensive, and samples a single location and depth that may not represent conditions along the track.

The result is a structural measurement gap: the layer that drives geometry degradation is the layer the standard inspection regime measures least directly. The question this paper addresses is how, and how far, artificial intelligence closes that gap.

2. Current State

2.1 The automated inspection foundation

Automated track measurement is now an established fact of the industry. Every Class I railroad uses some form of track geometry measurement system (TGMS), and FRA itself operates a fleet under its Automated Track Inspection Program (ATIP), conducting compliance surveys on over 150,000 miles of track annually. As of December 2025, the ATIP fleet includes three staffed rail-bound geometry platforms, two autonomous geometry boxcars, one autonomous geometry car operating in passenger service, a hi-rail geometry vehicle, and a hi-rail ultrasonic rail test vehicle; FRA notes that ATIP data supports machine learning as well as track safety trend analysis, state of good repair assessment, and risk analysis.

In October 2024, FRA proposed to revise its track safety regulations to require Class I and II railroads, intercity passenger railroads, and commuter railroads to operate qualifying TGMS at specified frequencies on mainline and controlled siding track meeting tonnage, passenger service, or hazardous materials criteria. In that rulemaking, FRA cataloged the technologies now used to measure track health, a list that includes ground penetrating radar, track imaging systems, machine learning based visual inspection of track components, vertical track deflection systems, and lidar 3-D scanning. The agency’s own inspection program already applies machine vision: algorithms trained to detect railroad infrastructure assets such as fasteners process images collected during ATIP surveys and identify possible defects, with software utilities that automatically extract high-resolution imagery around switches, frogs, geometry defects, and track with reduced lateral strength.

2.2 Substructure-specific sensing and AI

Three technology threads bear directly on the track base.

GPR with machine learning interpretation. GPR has been studied for railroad substructure assessment for more than two decades. Properly applied, it can observe ballast and subballast layer thickness, variations in thickness along the track, and water trapped in the ballast and soft subgrade; its principal historical limitations have been the unknown dielectric constant of ballast and the diffuse boundary between clean and fouled material. Machine learning applied to GPR signal features, in both time and frequency domains, has been used to predict ballast fouling indices directly from radar data, addressing the interpretation burden that previously required specialized technicians.

Deep learning computer vision for ballast condition. Research supported by FRA and conducted with the University of Illinois has progressed from machine vision evaluation of ballast fouling in the field, demonstrated under a Transportation Research Board Safety IDEA project, to a purpose-built Ballast Scanning Vehicle that uses multiple image acquisition devices to continuously capture scans of ballast cut sections, processed by a deep learning framework, replacing subjective single-point sampling with continuous, in-depth evaluation along the track. Successive algorithm generations have applied vision transformer architectures to ballast segmentation and condition classification, and researchers have addressed the training data bottleneck by constructing multi-dimensional ballast datasets that combine field imagery, laboratory imagery, and synthetically generated particles.

Multi-sensor data fusion. Recent work integrates GPR, satellite interferometric synthetic aperture radar (InSAR), and machine learning into a combined track health monitoring approach, pairing GPR’s view of fouling and moisture with InSAR’s rapid, network-wide diagnosis of settlement and subsidence. This fusion direction matters for the track base specifically, because substructure failure modes (fouling, drainage loss, subgrade deformation) express themselves across multiple measurement domains that no single sensor captures completely.

2.3 The regulatory and oversight context

FRA’s research strategy has explicitly identified the substructure opportunity: measurement technology and the data it collects can be used to understand critical track support and substructure behavior, determine root causes of track conditions, and predict future conditions and safe inspection intervals. At the same time, FRA research and development maintains a standing discipline that new inspection systems must be demonstrated to be at least as good as the methods they supplement or replace.

Oversight bodies have documented the adoption barriers. GAO, examining rail safety technologies, reported that experts saw uncertainty about effectiveness as a challenge for track inspection and measurement technologies in early stages of development, that railroads will not adopt a technology without confidence in a positive return, and that current regulations create a disincentive: new technologies identify defects perceived as too insignificant to pose a safety risk, yet once identified, those defects require remedial action. The Congressional Research Service, surveying freight rail safety for the 119th Congress, notes that the increase in the overall derailment rate from 2014 to 2023 appears driven largely by derailments on yard, siding, or industrial tracks, while the mainline derailment rate has remained close to or below 2013 levels. That distribution shapes the value case for substructure AI, since automated inspection deployments have concentrated on mainline track.

3. Analytical Framework

To structure the assessment, this paper organizes AI applications in track base management into three functional layers.

Layer 1: Perception. Converting raw sensor output into substructure measurements. This includes machine learning interpretation of GPR returns, deep learning segmentation of ballast imagery, machine vision detection of surface symptoms such as mud spots and fouled cribs, and automated processing of deflection and lidar data.

Layer 2: Condition assessment. Converting measurements into engineering judgments. This includes fouling index estimation, drainage condition classification, layer thickness and moisture mapping, and fusion of substructure indicators with geometry data to attribute geometry degradation to its subsurface causes.

Layer 3: Prediction and planning. Converting assessments into action. This includes degradation forecasting, undercutting and ballast cleaning prioritization, tamping cycle optimization, and risk-informed adjustment of inspection intervals.

The maturity of each layer differs, and the findings below are organized against this framework.

4. Findings

Finding 1: Perception-layer AI for the track base is technically demonstrated but unevenly fielded. Machine vision inspection of track components is operational within FRA’s own inspection program, and every Class I railroad operates automated geometry measurement. Substructure-specific perception, by contrast, remains concentrated in research and pilot deployments: the Ballast Scanning Vehicle and vision transformer condition algorithms are products of FRA-supported university research rather than fleet-standard equipment, and machine learning GPR interpretation is documented primarily in the peer-reviewed literature. The capability exists; the fielding lags.

Finding 2: The regulatory architecture measures the symptom, and the compliance value of substructure AI is therefore indirect. The 2024 TGMS rulemaking would establish automated inspection requirements for geometry, the surface manifestation of substructure condition, while listing GPR and machine learning visual inspection among available technologies without assigning them a compliance role. Until substructure condition indices, such as fouling classifications derived from GPR or imagery, acquire recognized standing in maintenance and safety standards, railroads capture their value only through internal maintenance planning, which weakens the investment case GAO identified as decisive for adoption.

Finding 3: Data is the binding constraint on Layer 2 and Layer 3 maturity. Deep learning condition assessment is intrinsically data driven, and the efficacy of these models is tied to the quality of annotated training datasets; researchers have resorted to synthetic particle generation to supplement scarce field annotations. FRA’s ATIP archive, which the agency already makes available to support machine learning and research, is the largest consistent public track measurement corpus and represents the most practical foundation for substructure model development at scale.

Finding 4: The human role is a design problem, and it is currently underdesigned. FRA-sponsored work on human-automation teaming in track inspection decomposes the inspection function into four tasks, data collection, data analysis, decision making, and action, and provides a design process for allocating those tasks between people and automation. Substructure AI raises the stakes of this design question, because its outputs (fouling indices, moisture maps, degradation forecasts) inform maintenance decisions rather than replicate what an inspector already sees, so the inspector’s role shifts from detection toward verification, interpretation, and remediation management.

Finding 5: The automation question remains contested, and substructure AI inherits that contest. [CONTESTED TERRAIN] Industry advocates automated inspection expansion with reduced visual inspection frequencies, citing detection performance; labor representatives and some members of Congress argue that technology and human oversight are complements and object to framing them as substitutes. FRA convened a Railroad Safety Advisory Committee working group on the subject; in October 2023 the group determined it would not be able to reach consensus, and the task was closed in March 2024 without a recommendation, after which FRA proceeded to rulemaking on its own record. This paper does not resolve that dispute. It observes that substructure AI is positioned differently from geometry automation within it: because substructure sensing measures what visual inspection cannot see at all, its strongest near-term case is additive capability rather than substitution for existing inspection.

5. Recommendations

For regulators and standards bodies. Develop recognized substructure condition metrics, beginning with fouling classification and drainage condition, that can be referenced in maintenance standards and safety analyses, so that AI-derived substructure assessments acquire standing beyond internal railroad use. Pair any such metrics with validation protocols consistent with FRA’s existing discipline that new inspection methods demonstrate performance at least equivalent to the methods they supplement.

For railroads. Prioritize substructure AI deployment on segments where geometry degradation recurs after surfacing, since recurring geometry defects are the classic signature of substructure root causes, and use fused GPR, imagery, and geometry data to shift ballast maintenance from cycle-based to condition-based programming. Structure pilot deployments to generate labeled data as a deliberate product, since annotated field data is the scarcest input to the next model generation.

For the research community. Concentrate on Layer 3: degradation forecasting and maintenance optimization models that consume the perception outputs now available, and on uncertainty quantification, so that model confidence can be communicated to the inspectors and engineers who act on AI outputs. Extend human-automation teaming design work specifically to substructure assessment tasks, where the automation contributes measurements no human baseline exists for.

6. Implementation Considerations

Three practical constraints will shape any deployment program. First, the GAO-documented remediation disincentive applies with particular force to substructure sensing: a technology that newly reveals marginal subsurface conditions creates obligations, or at least perceived obligations, that did not exist when those conditions were invisible, and program design should address remediation criteria for AI-detected substructure conditions before, rather than after, fielding. Second, the derailment data counsels honesty about where safety returns will materialize: with mainline derailment rates near or below 2013 levels and the rate increase concentrated in yards and sidings, the near-term case for substructure AI rests substantially on maintenance efficiency, asset life extension, and geometry defect prevention economics, with safety benefits accruing over longer horizons. Third, workforce transition planning should precede deployment, applying the human-automation teaming design process to define inspector roles in a substructure-instrumented environment.

7. Limitations

Public sources do not yet provide quantitative cost-benefit results for substructure-specific AI deployments at fleet scale, and this paper therefore makes no claim about the magnitude of net economic return; that gap is itself a finding about the state of the evidence. Vendor performance claims for commercial substructure inspection products were excluded as sole support for any conclusion, consistent with this paper’s sourcing standards. Detailed treatment of GPR antenna configuration and signal processing methods, discrete element modeling of ballast behavior, and the economics of undercutting versus renewal programs exceeded the scope of this paper and are candidates for subsequent publications in this series.

8. Conclusion

The track base has been the least measured and most consequential layer of the railroad. AI-enabled sensing has now demonstrated, in federal research programs and the peer-reviewed literature, that ballast fouling, layer condition, and moisture can be measured continuously and objectively rather than inferred from surface symptoms or spot samples. The remaining work is institutional: condition standards that give substructure measurements compliance standing, validation protocols that establish trust, shared data that feeds the next model generation, and a deliberate division of labor between algorithms and inspectors. The railroads and agencies that complete that institutional work will convert a demonstrated laboratory and pilot capability into a managed asset class, and will do so on the strength of evidence rather than assumption.

Bibliography

Tier 1: Government and Oversight Sources

Congressional Research Service. East Palestine, OH, Train Derailment and Hazardous Materials Shipment by Rail: Frequently Asked Questions. CRS Report R47435. Washington, DC, 2023. https://crsreports.congress.gov/product/pdf/R/R47435.

Congressional Research Service. Freight Rail Safety Issues in the 119th Congress. CRS Report R47911. Washington, DC. https://www.congress.gov/crs-product/R47911.

Federal Railroad Administration. ATIP: Automated Track Inspection Program (December 2025). TRB Quad Chart. Washington, DC: U.S. Department of Transportation, December 2025. https://railroads.dot.gov/sites/fra.dot.gov/files/2025-12/ATIP-TRB-QuadChart_Dec_2025.pdf.

Federal Railroad Administration. FRA Research, Development and Technology Strategic Plan, 2020-2024. Washington, DC: U.S. Department of Transportation, 2020. https://railroads.dot.gov/sites/fra.dot.gov/files/2020-07/Strategic%20Plan%202020-2024-A.pdf.

Federal Railroad Administration. Machine Vision Data Products for the Automated Track Inspection Program. Washington, DC: U.S. Department of Transportation, 2025. https://railroads.dot.gov/elibrary/machine-vision-data-products-automated-track-inspection-program.

Federal Railroad Administration. “Track Geometry Measurement System (TGMS) Inspections.” Notice of Proposed Rulemaking. Federal Register, October 24, 2024. https://www.federalregister.gov/documents/2024/10/24/2024-24153/track-geometry-measurement-system-tgms-inspections.

Federal Railroad Administration / John A. Volpe National Transportation Systems Center. Human-Automation Teaming in Track Inspection. Washington, DC: U.S. Department of Transportation. https://rosap.ntl.bts.gov/view/dot/66915.

U.S. Government Accountability Office. Rail Safety: Federal Railroad Administration Should Report on Risks to the Successful Implementation of Mandated Safety Technology. GAO-11-133. Washington, DC, 2010. https://www.gao.gov/products/gao-11-133.

Tier 2 and Tier 4: FFRDC-Adjacent, National Academies, and Peer-Reviewed Sources

Leng, Zhen, and Imad L. Al-Qadi. “Railroad Ballast Evaluation Using Ground-Penetrating Radar.” Transportation Research Record: Journal of the Transportation Research Board 2159 (2010): 110-117. https://doi.org/10.3141/2159-14.

Luo, Jiayi, Kelin Ding, Haohang Huang, John M. Hart, Issam I. A. Qamhia, Erol Tutumluer, Hugh Thompson, and Theodore R. Sussmann. “Toward Automated Field Ballast Condition Evaluation: Development of a Ballast Scanning Vehicle.” Transportation Research Record: Journal of the Transportation Research Board (2024). https://doi.org/10.1177/03611981231178302.

Luo, Jiayi, Haohang Huang, Kelin Ding, Issam I. A. Qamhia, Erol Tutumluer, John M. Hart, and Theodore R. Sussmann. “Toward Automated Field Ballast Condition Evaluation: Algorithm Development Using a Vision Transformer Framework.” Transportation Research Record: Journal of the Transportation Research Board (2023). https://doi.org/10.1177/03611981231161350.

Luo, Jiayi, Kelin Ding, Issam I. A. Qamhia, John M. Hart, and Erol Tutumluer. “Deep Learning Approach for Automated Railroad Ballast Condition Evaluation.” In Proceedings of the 5th International Conference on Transportation Geotechnics (ICTG) 2024, Lecture Notes in Civil Engineering 402, 49-57. Singapore: Springer, 2025. https://doi.org/10.1007/978-981-97-8213-0_7.

Selig, Ernest T., and John M. Waters. Track Geotechnology and Substructure Management. London: Thomas Telford, 1994.

Sysyn, Mykola, et al. “Advancing Railway Track Health Monitoring: Integrating GPR, InSAR and Machine Learning for Enhanced Asset Management.” Automation in Construction (2024). https://doi.org/10.1016/j.autcon.2024.105378.

Tutumluer, Erol, Narendra Ahuja, John M. Hart, Maziar Moaveni, Haohang Huang, Zixu Zhao, and Sagar Shah. Field Evaluation of Ballast Fouling Conditions Using Machine Vision. Safety IDEA Project 27 Final Report. Washington, DC: Transportation Research Board of the National Academies, 2017.