How to Build an Effective Checklist - Using Human Factors Principles

The aviation industry has long recognised the importance of checklists for enhancing safety and standardising procedures in flight operations. Over time, checklists have expanded to other domains such as medicine with the World Health Organisations Surgical Safety Checklist. Verbal aide memoirs have been used in diving for a while, e.g., ABC, BWRAF, GUE EDGE and over the last decade or more, written checklists have been developed for rebreather use as part of the build, pre-dive and post-dive checks.

However, checklist design and subsequent execution is not simply a case of writing a list of actions and expecting divers, instructors, or diving safety officers to follow them. Factors such as human performance limitations, the complexity and timing of diving operations, increased workload or time constraints, and the social environment, all need to be considered as part of checklist design and use. To address these challenges, the development of checklists should be based on human factors principles and designed for use under normal, abnormal, and emergency conditions. Most checklists in diving are based around normal operations, but some diving operations will have abnormal or emergency checklists as part of their risk/safety management protocols. 

Standardisation in checklist design is important, but overemphasis on standardisation can reduce the effectiveness of the checklist. As such, checklist design should balance standardisation with the need to adapt to different equipment configurations and specific situations. Compliance with standards and procedures is expected, but checklists should remind divers and instructors that standard operating procedures can be violated if necessary to ensure the safety of those involved in the diving activity is maintained.

Checklist length is a significant concern, as cognitive limitations can lead to errors when dealing with stress, concurrent task demands, and time pressure. A balance must be struck between including critical information to reduce memory load and maintaining a manageable length for timely completion. The appropriate level of detail and length should be evaluated on a case-by-case basis. A good rule of thumb would be 5-8 line items, each with no more than 5-8 words. If more words/line items are needed, then consider breaking the check into something smaller should be considered.

Typography (font, sizing, and spacing) is another important consideration in checklist design. Degani, a scientist at NASA Ames, recommends using sans-serif fonts for improved legibility and ensuring adequate spacing between lines of text to prevent skipping lines. For font size, a range between 0.14 and 0.20 inches (3.6 and 5.1 mm) is suitable for checklists, but practical considerations may necessitate adjustments. Font sizes below 0.10 inches (2.5 mm) are not recommended.

Phraseology (words, grammar, and tone) plays an essential role in the design of checklists for aviation. According to Degani and Wiener, guidelines for designing checklists include restricting vocabulary size, using phonetically balanced words, increasing sequential constraints between items, and employing frequently used words. The checklists should use plain language and should be a prompt or cognitive aid rather than a long instruction set.

Checklists should contain unambiguous responses. Degani and Wiener suggest that the response should convey the actual status or value of the item (e.g., configurations, pressures, or status). For example, calling out specific gas pressures or pO2 readings while checking SPGs or CCR controllers can provide verbal confirmation and aid mental preparation.

As humans, we have a limited capacity when it comes to short-term memory. As such, design techniques like chunking can help users group information into ‘concepts’. By structuring checklists into chunks, divers can more easily identify and follow the order of items e.g., for a rebreather, it might be about validating/calibrating the cells. In addition to chunking, other factors should be considered for the safe and effective ordering of checklist items, such as workflow or equipment layout e.g., bottom to top, left to right. Having a defined end-point and a positive confirmation that the checklist has been completed is also advantageous when operating as a team.

Decoupling is another aspect of checklist design that needs to be considered. In tightly coupled systems, where there is limited time or slack in the system, means there is a limited capacity for the divers to recover from a failure. This might be caused by the time between getting ready on the boat and jumping off the back of the boat when slack arrives early. Once ‘committed’ it is hard to say no, even if a fault has been detected as part of a checklist. Decoupling checklists to allow enough time (slack) for a diver to detect and recover from configuration failures is important. This might mean configuring the equipment a little earlier than normal.

Checklists are a valuable tool in dive operations, promoting mutual monitoring and coordination during high workload periods i.e., build checks at the dive site, or final checks before getting in the water. Their goal is to intentionally slow people down so that they move from System 1 to System 2 behaviours.

Error-producing conditions, such as time pressure, fatigue, interruptions, and poor surface conditions can affect diver performance and increase the likelihood that errors are introduced and not identified and removed. Because of the way our brains work, interruptions to checklists can lead to items being skipped because we think we are further down the checklist than planned. Some rebreathers have electronic checklists which can help mitigate this issue by allowing checklist progress to be followed. Furthermore, by linking electronics with sensors e.g., HP O2 or diluent sensors, progress can be correlated with sensed data.

Given the lack of a formal reporting system in diving and an absence of a Just Culture, it is difficult to quantify the number of errors made while executing checklists, how often non-compliance to checklists occurs, or when checklists have provided a ‘save’. All of these make it hard to determine the effectiveness of checklists and why checklists are or are not used.

Summary

Checklists are not a panacea to safe diving operations. They need to be designed with human factors principles in mind, and they require a certain social construct to exist before their use is normal. They work best within teams who can hold each other accountable to the standards set by the team.

If a checklist is not used, and a successful dive happens, it can start the normalisation of deviance/risk because the outcome was good, despite the process being flawed.

Checklists should not be developed for the sake of running a checklist. A checklist builds on technical skills that have been acquired during training and consolidation diving, they should not be used as a liability limiting/transfer device so that if something goes wrong, the lack of checklist adherence can be used as the reason for the failure. If a checklist is consistently not trapping errors, the system, including the checklist, needs to be examined to see how it is making sense not to use the checklist. If multiple people are failing, it isn’t an individual ‘human’ issue, it is much bigger. 

Part 2 of this short series on checklist design can be found here although they weren't written as part 1 and part 2, it just happened that way!!

References:

Degani, Asaf; Wiener, Earl L. Human Factors of Fight-Deck Checklists: The Normal Checklist. Contract No. NCC2-377. A report prepared for Ames Research Center. May 1990.

Degani, Asaf. On the Typography of Flight-Deck Documentation. Contract No. NCC2-327. A report prepared for NASA. December 1992.

Burian, Barbara. “Design Guidance for Emergency and Abnormal Checklists in Aviation.” In Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting. 2006.

Gareth Lock is the owner of The Human Diver, a niche company focused on educating and developing divers, instructors and related teams to be high-performing. If you'd like to deepen your diving experience, consider taking the online introduction course which will change your attitude towards diving because safety is your perception, visit the website.