Dom Cooper analyses two types of incident rates: lost-time- and total recordable- and questions their validity.
While many view safety metrics as key to improvement, given their use in gauging the effectiveness of an entity’s safety management, some are calling for summary Lost-Time Incident Rate (LTIR) and Total Recordable Incident Rate (TRIR) to be abandoned; it is felt they are wrongly used for blame and incentivising managerial performance, easily manipulated, influenced by under-reporting, and do not predict serious injuries and fatalities (SIFs). Interested to see if these claims stood up to scrutiny, I did some digging.
CREDIT: Jason Tuinstra/Unsplash
Typically, LTIR/TRIR are calculated to facilitate benchmarking [[i]]. These ‘lagging indicators’ use the number of recordable lost-time incidents occurring per number of hours worked over a designated accounting period. A recordable incident essentially relates to the severity of outcome, including loss of consciousness, days away from work, restricted work, ‘managed’ transfer to another job, certain medical treatment cases, diagnosed industrial disease cases and certain ‘dangerous occurrences’ (near-miss incidents with the potential to cause harm). The UK tends to use LTIR (or variants of), while the USA uses TRIR; both reflect the number of recordable incidents per 100 full-time workers (i.e., 200,000 hours) during a year. They differ by the scope of incidents included; LTIR solely reflects the number of lost-time cases, whereas TRIR also includes restricted work and medical treatment cases. Lower rates implicitly signal better overall safety performance.
However, LTIR/TRIR calculation is affected by the number of employees and the hours worked; incident rates increase consistently as firm size decreases [[ii]]. This is unfortunate, as LTIR/TRIR is often used as a performance comparator between small and large entities to award contracts (e.g., in Oil & Gas and Construction). OSHA explicitly frowns on using TRIR in this way [i], even providing company-size categories to facilitate like-for-like comparisons.
Blame: LTIR/TRIR and other lagging indicators are used to hold around 1 in 5 occupational safety & health (OSH) professionals to account for their organisation’s OSH performance, even though much of that is usually outside of the OSH person’s direct control [[iii]]. This blame may adversely influence the reporting and classification of incidents by unethical suppression.
Incentives: LTIR/TRIR metrics are often used as the basis for safety incentive programs: if the rates trend downwards, rewards follow; if they go up, potential rewards are lost. However, linking incentives to reportable injury rates can have unintended or ‘perverse’ consequences, such as manipulation of injury classification and/or underreporting stemming directly from pressure not to report [[iv]].
Manipulation: TRIR is dependent on the number of recordable incidents, but there is a thin line between what is deemed a medical treatment case (recordable) or a first-aid case (non-recordable), and this grey area can be, and often is, manipulated to downplay an entity’s incident history [[v]].
Under-reporting: Estimates based on the difference between the Labour Force Survey (LFS) and the data associated with the Reporting of Injuries, Diseases and Dangerous Occurrences Regulations 1995 (RIDDOR) suggest workers themselves do not report approximately 57–80% of all incidents [[vi]]. Similarly, entities under-report recordable incidents to the regulator by around 50% [[vii]]. Of those reported, a 2015 British HSE study [[viii]] showed that 10% of RIDDOR major injury cases were less serious, while 25% of injuries classified as less serious were actually major injuries. Thus, some appear confused about what constitutes a major injury, some perhaps lack awareness of an incident’s facts and some may have reason to want to downplay its seriousness (e.g., higher insurance rates for a higher incident rate). The LTIR/TRIR, therefore, of many entities may seriously underestimate the true scale of their OSH problems.
Prediction: It is debatable whether LTIR/TRIR are predictive of SIFs or future safety performance. Some research suggests not, as neither LTIR/TRIR are sensitive to the severity of incidents simply because they combine lesser and major injuries into one metric [[ix]]. While lesser injuries tend to swamp the number of major injuries within any LTIR/TRIR calculation, they won’t predict SIFs due to their different precursors [[x]]. For example, a 2003 Australian mining study revealed that fatalities persisted while lost-time injuries simultaneously declined [[xi]]. Similarly, in 2005, the Oil & Gas Producers Safety Committee concluded that LTIR/TRIR had little predictive value toward the potential escalation to single and multiple fatalities or major accident risk [[xii],]. Such findings make perfect sense when LTIR/TRIR are considered as measures of productivity rather than safety; i.e., they reflect the number of incidents or events that resulted in lost productivity (workdays) due to work-related health and safety system failures [[xiii]]. Conversely, using a very large dataset, recent US National Institute of Occupational Safety and Health (NIOSH) Mining research provides strong evidence that traditional lagging indicators can be used to predict future levels of fatal events [[xiv], [xv]].
LTIR/TRIR metrics should be taken with a large pinch of salt given that they are highly susceptible to bias. These biases are primarily concerned with misclassification of the severity of incidents and under-reporting, which means the metrics are unreliable (i.e., they lack consistency). If a measure is unreliable, it must also be invalid, as the two are inextricably linked [[xvi]]. It is doubtful, therefore, that they adequately reflect actual safety performance. Thus, using them for managerial incentives or to assign blame to OSH professionals for poor performance seems misguided; incidents are often outside of the direct control of either OSH professionals or line managers. However, events can be influenced by appropriate interventions, resources and effective leadership, and it is these that should be incentivised. The notion that LTIR/TRIR are not predictive of SIFs is not as clear-cut as some would claim; there is strong evidence for and against, signalling that the ‘jury’ is out and much more research into the prediction issue is required.
If entities are to continue using LTIR/TRIR, the OSH profession must reflect on how to improve their reliability while simultaneously decreasing classification manipulation and increasing incident reporting. Monitoring actual and potential SIFs as an alternative or complementary outcome metric should also be considered (e.g., (actual SIFs or potential SIFs / Total # Hours) x 200,000).
[i] OSHA (2016). Clarification on how the formula is used by OSHA to calculate incident rates. https://www.osha.gov/laws-regs/standardinterpretations/2016-08-23
[ii] McVittie, D., Banikin, H., & Brocklebank, W. (1997). The effects of firm size on injury frequency in construction. Safety Science, 27(1), 19-23.
[iii] Minnick, W. D. & Wachter, J. K. (2019). The role of leading and lagging indicators in evaluating OSH professionals’ performance. Professional Safety, 64(01), 32-36.
[iv] McDermott, V., Zhang, R. P., Hopkins, A., & Hayes, J. (2018). Constructing safety: Investigating senior executive long-term incentive plans and safety objectives in the construction sector. Construction Management and Economics, 36(5), 276-290.
[v] Welch, L. S. & Hunting, K. (2003). Injury surveillance in construction: What is an “injury”, anyway?. American Journal of Industrial Medicine, 44(2), 191-196.
[vi] Health & Safety Executive, (2004). Health and Safety Statistics Highlights 2003/04
[vii] Probst, T. M., Bettac, E. L., & Austin, C. (2019). Accident underreporting in the workplace. Increasing Occupational Health and Safety in Workplaces, 30-47.
[viii] Health & Safety Executive, (2015). Evaluation of trends in RIDDOR reportable injury data reported to HSE by duty holders pre- and post- change to over-7-day reporting. RR1054, HSE Books
[ix] Pirzadeh, P., Lingard, H., Benson, A., & Alderuccio, J. (2023). Measuring and managing health and safety performance on major rail infrastructure construction projects. Digital Transformation of Health and Safety in Construction, 349-358.
[x] Wachter, J. K. & Ferguson, L. H. (2013). Fatality Prevention. Professional Safety, 58(7), 41-49.
[xi] Towsey, C. A. J. (2003). Proactive Measures for Fatality Prevention in the Mining Industry—Why Fatalities Persist While Lost Time Injuries Decline. Mining Risk Management, 165-174.
[xii] Zijlker, V. (2005, March). What are the major health, safety and regulatory issues and concerns in worldwide operations. In International Regulators Forum Offshore Safety Forum.
[xiii] O’Neill, S., Martinov-Bennie, N., Cheung, A., & Wolfe, K. (2013). Issues in the measurement and reporting of work health and safety performance: A review. Sydney: Macquarie Lighthouse Press
[xiv] Yorio, P. L. & Moore, S. M. (2018). Examining factors that influence the existence of Heinrich’s safety triangle using site-specific H&S data from more than 25,000 establishments. Risk Analysis, 38(4), 839–852.
[xv] Yorio, P. L., Haas, E. J., Bell, J. L., Moore, S. M., & Greenawald, L. A. (2020). Lagging or leading? Exploring the temporal relationship among lagging indicators in mining establishments 2006–2017. Journal of Safety Research, 74, 179-185.
[xvi] Axelson, R. D. & Kreiter, C. D. (2009). Reliability. In Assessment in health professions education (pp. 77-94). Routledge.
For sure best not to measure, record or report or disclose predictable known issues as then you may have to do something about them retrospectively………
https://www.hse.gov.uk/research/rrpdf/rr561.pdf
https://icd.who.int/browse10/2016/en#/H53.1
https://www.youtube.com/watch?v=2C1jmwGIsGQ&list=PLezLOQBs0kcn1kCE3A_Jr5eShBiLu3kKy&index=4