How to leverage technology for policy and procedure development
Strong safety policies and procedures are the foundation of a thriving safety culture and a robust safety management system. They set clear expectations, foster compliance, define roles, and provide benchmarks for audits (Dekker, 2012). Yet, it is all too common for policies and procedures to be lacking, ineffective, or obsolete. While companies and leaders recognize this challenge, they often lack the time and resources to address it effectively. Here is where AI comes in. Imagine enhancing and streamlining policy and procedure development with artificial intelligence (AI). This article explores how to leverage AI to develop and review safety documentation, while acknowledging its limitations and the critical role of human oversight.
The Challenge of Effective Policies and Procedures
Having effective safety policies and procedures is difficult. Controlling risk requires a comprehensive understanding of potential hazards and their mitigation measures, yet policies are often written by engineers and policy writers without worker involvement and thorough understanding of the actual tasks being done (Hollnagel, 2017). This creates a significant gap between work-as-imagined and work-as-done. It is also common for written guidance to be vague or ineffective, creating various error drivers such as field decisions, difficulty in execution, confusing information, or multiple or embedded steps.
Readability is paramount; policies and procedures must be easily understood by all employees, regardless of background or education (Hale & Borys, 2013). Even when documentation is effective, organizations often end up with thousands of procedures, making it impossible for workers to remember them all or find them when needed.
Using AI for Policy and Procedure Development
When developing safety documentation with AI, understanding your desired structure is crucial. A well-defined structure ensures consistency, clarity, and completeness, making policies and procedures easier to understand, implement, and enforce. When utilizing AI to support development, it is critical to specify the requirements for each document's content and structure, including:
- Policy Title and Number: Clear and concise identification
- Purpose: Explaining rationale and articulating goals
- Roles and Responsibilities: Critical for accountability, though AI needs specific guidance here
- Definitions: Technical terms need careful attention to maintain intelligibility
- Content: Detailed step-by-step instructions, decision criteria, risk controls, and verification steps requiring clear, concise, and precise language to ensure usability and compliance in the field
- Exception Management: AI can efficiently identify recurring exceptions across documents, standardizing responses and ensuring consistency while allowing human oversight to handle unique or complex cases
- Training Requirements: AI can identify needs but may need help specifying components
- Document References: AI excels at creating comprehensive reference lists and indexes
AI Use Cases in Policy and Procedure Development and Consultation
Specialized AI tools like Google NotebookLM can revolutionize how we develop and consult procedures. For instance, we utilized it to assist in developing an ISO 45001 compliant management system for a plastics manufacturer (ISO, 2018). By feeding OSHA, ANSI, and ISO standards into the tool, the AI generated draft policies that were generally of good quality and covered most compliance elements. While the AI significantly reduced policy writing time—by approximately two-thirds in this case—time was still required for review and verification, given some issues with AI "hallucinations" (fabricating information) and the carryover of information from unrelated policies. AI also struggled with defining role-specific "shall" statements, requiring more general input about required actions.
By uploading relevant safety documentation, standards, and procedures into various instances of NotebookLM, another organization created a reliable knowledge base for consultation. This process, known as Retrieval Augmented Generation (RAG), significantly reduces AI hallucinations by anchoring responses in actual source documentation. Additionally, the system provides a numbered link to each reference, allowing users to verify the original source firsthand, ensuring greater transparency and accuracy in safety documentation.
Another company leveraged AI in safety procedure development by using Otter.ai to record a conversation with a machinist operating a CNC milling machine in a manufacturing plant. The machinist detailed the process of producing precision metal components, covering setup, tool selection, speed adjustments, quality control, and shutdown. The recorded conversation was then fed into ChatGPT alongside existing facility procedures. ChatGPT synthesized the information into a well-structured first draft, aligning it with company documentation. This process captured expert knowledge accurately and ensured consistency across procedures. AI's ability to integrate real-world input with best practices enhanced procedural clarity and usability. However, a human reviewer was essential to refine contextual nuances, confirm regulatory compliance, and validate steps against actual conditions, ensuring a practical and fully compliant procedural guide.
Evaluating and Refining Documentation with AI
AI can help evaluate policy and procedure drafts by analyzing large amounts of text, identifying patterns, and comparing against best practices and regulatory requirements. It can quickly identify documentation strengths and suggest improvements, such as more specific training metrics or clearer guidance on handling non-compliance.
AI can also analyze procedures for common error traps, including:
- Field decisions without clear criteria
- Mental and physical difficulty in execution
- Vague or confusing information
- Conflicting or inconsistent instructions
- Multiple or embedded actions hidden in notes and warnings
Understanding AI's Limitations and Best Practices
While AI offers powerful capabilities, it's crucial to understand its limitations. AI can make mistakes, including:
- Hallucinations: Fabricating information or making incorrect connections
- Lack of Context: Missing nuances of specific workplace situations
- Over-Generalization: Producing generic content that doesn't address specific hazards
- Inconsistency: Creating policies with internal contradictions without proper guidance
Even the most seasoned safety professionals and meticulously researched policies can inadvertently overlook crucial details or misinterpret complex regulations. The sheer volume of information coupled with the potential for nuanced interpretations makes it challenging to guarantee complete accuracy and comprehensiveness. It is precisely through human-AI collaboration that meaningful improvements in this area can be achieved, combining AI's efficiency with human expertise to enhance clarity, precision, compliance, and usability.
Post-AI Development Checklist
After drafting documentation using AI, it is critical to:
- Check for regulatory compliance: While AI can greatly aid in evaluating that requirements are met, it may misinterpret regulations or miss recent updates. Human review is essential.
- Refine with human expertise: AI is a tool; human oversight ensures content is appropriate for your specific context and effectively controls risk.
- Validate with workers: Ensure procedures reflect actual work practices and are clear to those who will use them.
Looking Ahead
AI is transforming how we approach safety policy and procedure development, making it more efficient, consistent, and effective. By leveraging these tools thoughtfully while maintaining human oversight, organizations can create safer workplaces.
While this article focuses on policy and procedure development, AI's potential in safety management extends further. Future articles will explore practical applications in areas such as risk assessment, training development, and incident investigation. Each of these applications requires careful consideration of both AI's capabilities and limitations to ensure effective implementation.
Coming Soon: In our next article, we'll explore how AI is transforming incident investigation and root cause analysis. We'll examine practical applications of AI in analyzing incident data, identifying patterns, and generating insights for prevention.
References
Dekker, S. (2012). Just Culture: Balancing Safety and Accountability (2nd ed.). Ashgate Publishing.
Hale, A., & Borys, D. (2013). Working to rule, or working safely? Part 1: A state of the art review. Safety Science, 55, 207-221.
Hollnagel, E. (2017). Safety-II in Practice: Developing the Resilience Potentials. Routledge.
ISO. (2018). ISO 45001:2018 Occupational health and safety management systems — Requirements with guidance for use. International Organization for Standardization.
