Do you have a holistic and clear picture of what your Company’s risks (and thereby the liabilities) are? What is the company concerned about, and what is of less concern?

Do you have a layered risk management strategy that ranges from prevention over mitigation to response?

Have you analyzed and developed your “Risk Library” that builds the framework for the scope of the risk assessments? As a producer, how many different risk drivers do you have? Market Risk, Operational Risk, Strategic Risks, Insurance Risk, Reputational Risk, … and more? Are you skilled enough and confident in risk identification, assessment, evaluation, mitigation, performance measurements, communication and reporting?

Do you understand how human factors and company culture plays into the complex picture of risk assessments?

Who manages these risks in your production facility, who are the “Risk Owners” responsible for the assessments, the risk evaluation and mitigation measures and the final performance evaluation? Most likely that will be different persons depending on whether the risk evaluation is focused on product safety, environment, or experimental animals welfare and correct test results for release of vaccine batches. How do these different risk owners work together? Do they use the same systematic tools and systems? Could they? Should they?

The risk assessment analysis for a specific risk provides the foundation upon which the production unit / the company / the institution rests. It should be a document that is easy to read and understand. A good assessment is well defined and clear in scope. It will identify the types of risk being addressed: Product, personnel, animals, surrounding community, environment, economical, company image etc. It will be very specific in defining where this assessment is applicable and where it is not.

The more disciplined the writing process is around a risk assessment, the easier it will be for the institution to produce the necessary amount of risk assessments. If written in a disciplined manner there are sections and paragraphs that can be reused in “brother and sister” assessments.

A COLLABORATIVE EFFORT

Assembling an experienced, knowledgeable team is probably the single most important element in conducting a successful risk assessment. Individuals experienced in the GMP process steps, design drivers, operation, and servicing of equipment, and utility systems or facilities are essential. Team members usually include production, operating and maintenance personnel, design and/or operating engineers, specific skills as needed (product purity, QA/QC, waste handling, chem/bio, structural engineer, radiation expert, depending on the topic) and safety representative(s).

Knowledge of design standards, regulatory codes, past documented and potential future operational errors, accidents and incidents as well as maintenance challenges brings a hands-on reality to the review. 

Any risk assessment done by one single person will most likely not cover all relevant aspects. That is also the reason that risk assessments cannot be outsourced to a consultant. A consultant can challenge the conclusions, review the assessments, but not drive the process, manage the data, and keep the focus for the group. That has to come from the inside.

A STEP-WISE APPROACH

A comprehensive risk assessment consists of a series of consecutive steps, starting with assessment. The five steps are:

Some textbooks talk about AMP: Assessment, Mitigation and Performance. While this is correct, there is a very important evaluation step that is often overlooked, dismissed or forgotten. That is unfortunate, as the evaluation step is a crucial intermediate step in deciding what should be mitigated and what should not. Likewise, the corrective action plan (CAP) is an important tool that helps the institution to manage and communicate what is being mitigated, who is responsible for the individual steps, and within what time-frame can a result be expected, which will then need to be performance verified.

HAZARDS & THREATS

Risks arise from hazards or threats. A hazard/threat is a well defined thing. When a hazard is exposed to a procedure or an action it becomes a risk.

An agent is a hazard, and so is a chemical. A high pressurized steam system, an 80 degree C WFI system, asphyxiants used for propagation in fermentors are also hazards. As long as the system or the hazard is left alone, it is not a risk. If we handle an agent, even from a low risk group in our production process, propagate it in a fermentor, spin it around in a centrifuge, it can become a risk.

A car is a hazard. As long as it is parked, all is good. If somebody performs an action and drives into the parked car it can become a risk. If we drive the car and it hits something it becomes a risk. If we use it outside its intended use and attempt to cross a river, it becomes a risk. If we leave a kid in the car during a heat wave – it becomes a risk.

IMPACT

SCOPE

Another important part of performing an effective analysis is identifying the boundaries of the review and having the right information available.  The boundaries of the review may be a single piece of equipment, a collection of related equipment, or an entire vaccine facility with associated QA/QC laboratories and sample transport in between.  

A narrow focus results in an analysis that is more detailed and explicit in identifying the hazards, describing the risks, and specific recommended controls. As the review boundaries expand to include the equipment involved in a large complex production process or even an entire production facility, the findings and recommendations cannot be as detailed as they would for a more narrow process. They will then become observations and conclusions on more of an overview level.

The boundaries can include the steps in the construction of the system under review, the steps involved in the operation of the equipment or facility, or the steps required to maintain the equipment or facility.  

A clear definition of the boundaries of the analysis will start the review off in an effective manner and will also begin to inform the choice of risk analysis methodology.

INFORMATION

All risk assessments have the aspect of information gathering.  One important way to gather information on an existing process or piece of equipment is to visit and walk through the operation. Videotapes or photographs of the production operation or maintenance procedures are important and often overlooked as excellent sources of information and documentation. Additionally, design documents, operational procedures, and maintenance procedures are essential information for the review team.  

If these listed documents are not available, the review team should develop them in the process. Effective reviews cannot be conducted without up-to-date, reliable documentation. 

PROCESS

When the necessary documentation has been identified and brought forward, the next step is to conduct the actual analysis.  It is helpful if an experienced facilitator leads the group through the process. The process is similar to all other brainstorming processes. Post-it notes, mind-maps, Murals and similar work spaces are helpful. Whatever works for the team that they are comfortable with should be used, as long as an experienced facilitator is guiding the process and someone else is capturing the data. The facilitator does not necessarily need to be a senior subject matter expert, it is more important to possess good skillset of managing the room and the group, create a structure and discipline throughout the process. The meetings need to be reasonably short. Long meetings can result in the team members losing energy, burning-out, and beginning to surface skate to finish the analysis rather than probing deeper.  

EVALUATION & MITIGATION

When the scenarios have been described and the risks identified, it is time to evaluate the identified risks and decide whether they are acceptable or not. The not acceptable risks needs to be identified, prioritized and mitigation measures needs to be decided upon.

Discuss the results of the risk assessment

• In your opinion, are the results logical?  Why/why not?
• Which sub-procedures are driving the overall risk scores?

Determine acceptability of the risks

• Considering the information specific to each of your products
  • Which procedure steps are acceptable seen from a GMP product or cross contamination perspective? 
  • Which risks aren’t acceptable? 
  • Why/why not?
  • Mark the procedure steps that can be accepted and those who cannot. Get approval from QA/QC
  • Write SOP
  • Train and retrain staff

RISK COMMUNICATION

If the risk assessment has been adequately comprehensive for facility design or SOP writing it will now be a very big and complex document. The results needs to be presented in such a way that the general person can keep the overview and understand the message. 

Sometimes, inspecting authorities will want to see the actual risk assessments. In theory, providing a 72 page printed excel document with 19 columns and 31 rows per page will meet the request from the GMP or the environmental health and safety inspector. It will not necessarily facilitate the discussion and hopefully successful certification that will follow. Communicating the risks in a clear and comprehensive manner is a large and difficult task. It takes time and the correct experience and tools to sort and visualize the data.

Risk = f (likelihood, consequence)

Risk can be visualized in a two dimensional graph. Likelihood and consequence on the x and y axes.

Usually, the department head, production managers, engineering lead or plant manager is the customer of the review. The facilitator/lead for the review team will generate an executive summary that details the scope of the review as well as the major findings and recommendations. The report recommendations can include who has been assigned the responsibility to follow up and a time frame. A separate staff member or function will review the final recommendations and determine the actual actions, responsibilities and deadlines. A periodic report can then generated to summarize the present status of each of the recommendations and progress. 

MITIGATION

The risk assessment team will not only have assessed the risk at each situation but has also made their recommendations for each situation, as the brainstorming and discussion naturally leads to the recommendations. The team will then continue the review question by question until the entire process or operation has been analyzed. From time to time it is beneficial to stop and look at the big picture to determine if the team have inadvertently missed anything.

Low and high hanging fruits

• What mitigation measures are needed?
• What can be applied easily?  Quickly?
• What needs to be implemented in the future?
• What needs to be presented to management (permissions/legal….)  before implementation will be possible?
• What will require investments and presentation for management before implementation will be possible?

List mitigation measures in logical order(s) (including estimated resource allocation and suggested milestones) for upper management to review, understand and decide.

CORRECTIVE ACTION PLAN (CAP)

Create an action plan to reduce the risks

• Considering all the generated information and final decision from upper management, develop a plan for the implementation of mitigation measures.
• Include deadlines, responsible person, etc.
  • If using a computer based tool, consider to include before/after graphs to illustrate how the risk will change if mitigation measures are implemented as intended.

PERFORMANCE EVALUATION

After implementation of the chosen risk mitigation measures these must be verified for intended performance. This is a task that obviously should be done right after implementation. However, it is important to understand that human factor plays a very important role for successful risk mitigation. Sometimes the chosen solution or engineering control will invoke a new problem that creates new risks in the same or in a different area. This means that the problem just moved and is now present in another form somewhere else. New mitigation measures that are introduced benefits from a 3 month, 6 month and 1 year follow up. Not all procedures are robust and durable enough to survive as times goes by. The human factor plays a very large role with regard to this matter.

Efficiency of mitigation measures, is the risk reduced?

• Use Performance Indicators and evaluate and verify that the mitigation measures actually worked as it was intended.

Make plans for Review and Validation

• Schedule a Risk Assessment Plan for when to review the existing plan and mitigation/performance
• Review after incidents or product complaints/recalls
• Review after facility changes or upgrades
• Review after new procedures or new agents are added to current production activities

If the final conclusion is the risk is still too high, then a stand down might be the only choice until another acceptable mitigation strategy has been identified.

TIMING

Risk assessments should be performed before initiating work, when a new procedure or process is introduced, after changes in procedures or staff health conditions, or when a facility design project is initiated. A risk assessment is a snapshot in time and should be followed up with timely reviews, verifying that the process, the attitude, and behavior from staff or other aspects has not changed. A new risk assessment should be performed after introduction of new agents/chemicals/procedures, after laboratory or utility renovations, and in relation to major accidents or incidents .