How to Quantify the Qualitative: Making EM Measurable - 2020 Rave Summit Recap

In October, the 2020 Rave Virtual Summit brought together critical communications and safety experts from across industries. These professionals shared best practices, innovative strategies, and important lessons learned about keeping organizations and communities both safe and informed both during emergencies as well as day-to-day operations.

Erica Hupka, Assistant Director of Emergency Management and Business Continuity for the University of Kansas Health System, led a session on October 21. Hupka’s session was titled “How to Quantify the Qualitative - Making EM Measurable Through Rave”, and detailed the process her organization improved emergency planning and boosted community engagement by identifying metrics and leveraging data. 

What Emergency Managers Can Learn From Metrics

Hupka began the session by defining the emergency management profession, saying much of the work is based on soft skills, such as the ability to network, educate, write an effective exercise, or appeal to people to "buy into" an emergency preparedness program. Many organizations struggle to determine the value of a response tool - unless there’s a disaster, emergency managers do not see an immediate measurable return on investment. When looking to emergency management for business excellence, it’s crucial to look at all the various metrics it’s possible to track, and how these might tangibly improve processes. 

What Is the “Baldrige Excellence Framework”? 

How does an administrator know their emergency exercise program is effective? The Baldrige Excellence Framework provides a way to evaluate programs - the system requires leaders to create a Criteria for Performance Excellence, establish core values and concepts, and create guidelines for evaluating processes and results. 

The first step is to identify the organization's core values, mission, and vision, as well as how a business plan can come together around these goals. This starts with leadership, strategy, and customers, and how those integrate with your operations and workforce to achieve results. 

Related Blog: "Communication Problems": What Really Goes Wrong - 2020 Rave  Summit Recap

The most challenging element of the framework is determining metrics with results that matter. There are many options for quantifying results - but how do you know what you’re quantifying is effective? How do you know what you're quantifying is useful information, and not just busywork? 

Hupka, along with a team of four, spent months determining which results would provide value and analysis to improve programs for both workers and stakeholders. The Baldrige Excellence Framework, which is available in a step-by-step document online, provides a great starting point for evaluating an organization's current position and making a better sense of potential metric determination. 

How To Determine Effective Metrics? 

Hupka explains that the most important metric the team identified was networking - understanding who to call, when to call them, how to get a hold of them, and ensuring all these contacts are available during an emergency to facilitate response. The team considered measuring networks by the number of people or contacts included, or various organizations the team interfaced with but decided these numbers were too static. 

Instead, the team decided to measure the number of times these networks met or did an activity to measure time investment. Once they measured meetings, they realized these metrics did not offer data to leverage for improvement either. 

exercise cycleThen, the organization knew they needed a new emergency operations plan, and began to explore relevant metrics related to safety planning. First, Hupka measured how many pages or appendixes were documented, which was tangible. However, this didn’t have a benefit to make the program more robust. Instead, the team decided to measure the number of exercises, actual events, improvement plans, and how those improvements are being tracked, addressed, or resolved using an “Exercise Cycle” chart. Following up on improvement items strengthened networks within key stakeholders; it allowed facilities to do facilities work, HR to write HR policies, and everyone to better understand how these subsets of the business function. 

Related Blog: The Next Phase of COVID-19 Response and Beyond: 2020 Rave Summit  Recap

These processes and planning improvements made the agency more resilient. The team established “Follow-up Fridays” to solve issues or conflicts among stakeholders or business subsets. However, the team decided the metrics were still not yielding adequate results - and returned to the drawing board with an auditing team. 

The most simple thing the auditing team identified was - how do you communicate with students or stakeholders? The answer was through a comprehensive mass communication and collaboration platform, which opened the opportunity to track a variety of metrics, including frequency of text messages or emails, and incidents where a community member received or didn’t receive alerts. 

By doing so, safety managers were able to team with the IT team to improve message delivery. The team was able to create conversations around the best way to communicate with key customers, students, and staff, as well as the type of message these parties needed from these alerts. These results enabled a dialogue - and helped the program boost awareness, determine how best to use two-way communication, initiate branded conversations from emergency management and humanized emergency response groups. 

How Communications Data Improved Safety Perceptions

Data from a campus safety app also allowed dispatchers to measure how frequently text tips were coming in and the kind of incidents tool was being used for. This evaluation process completely revamped communication strategy - afterward, emergency managers put posters on campus and led lectures to raise awareness for the apps safety timer and two-way texting. Overall, student confidence in clear and timely alerting improved.

Related Blog: #SHOWUP: Defining the Standard of Excellence in Your PSAP 2020  Rave Summit Recap

Administrators were also able to identify students' most pressing fears or needs and were able to better allot safety resources. For example, by tracking requests for a virtual escort in the low-lit parking lots on campus, the agency was able to identify a need for better light fixtures in certain areas. This also improved community relationships, with not only students but also staff and workers from a nearby hospital felt safer coming on campus following the lighting update. 

Hupka also gave annual surveys to students or workers to analyze how safe everyone felt. Over two years, the university improved perception of safety by from 85% to 95%, due to the changes made by the campus safety department. The university was able to leverage our solutions to make data-driven changes to emergency response plans and boost overall safety perceptions.

Rave Summit Quantify the Qualitative

Mary Kate McGrath
Mary Kate McGrath

Mary Kate is a content specialist and social media manager for the Rave Mobile Safety team. She writes about public safety for the state & local and education spheres.

Introducing Rave Collaborate for Tactical Incident Management

This central platform helps you manage your organization’s preparedness, response, recovery and mitigation needs.

Learn More Here>

Schedule a Free Consultation

Talk With An Expert

Discover our pre-packaged solutions or configure a package that's right for your business. Learn how you can be up and running in days, take advantage of unlimited usage, and benefit from unbeatable performance and customer satisfaction.