The rural health system is focused on the relationship between patient flow and staffing levels amid the nationwide nursing shortage. Nowak said the analytics department deployed a model to determine labor allocations based on bed utilization.
“The idea is getting the right person at the right place at the right time,” he said.
Nowak added that the health system monitors indicators like staff turnover and overtime, along with responses from employee surveys, to measure the success of their operations models.
“If you can keep an employee who’s much more fulfilled, enjoys coming to work, has fun at work—they’ll be happier and our patients will have better outcomes,” he said.
Avoiding the ‘black box’
The technology would be obsolete if it weren’t for constant monitoring of efficacy, reliability and bias. Predictive models can be less accurate for vulnerable populations and can exacerbate existing health disparities. That’s why it’s important for developers to understand every model in totality—and to grasp that analytics tools are just aids, with no autonomous decision-making capabilities, Nowak said.
The technology “is giving a provider one more tool in their toolbox,” he said.
“When you get into these algorithms that are being developed, you hear about the infamous ‘black box,’ ” he added. “It’ll spit out some predictions and you say, ‘Well, that’s what the computer said.’ ”
To prevent that from occurring, hospitals subject their models to a variety of stress tests.
Metsker said hospitals must also use sample sizes that reflect the populations to which they’re applying predictive models. Otherwise, the models will carry a bias.
“If you’re talking about a clinical model, those variables have to be validated with a reasonable population. A hospital in the middle of Chicago has a different population than a hospital in Gig Harbor, Washington,” he said. “On the operations side, you have your different set of variables [depending on location]—volumes, services, geography and finance.”
Once the models are deployed, analytics leaders say they are checked in real time for accuracy and compared with historical data to track longitudinal efficacy.
“I always remind people of how we go about doing this,” Marroquin said. Any decision to use analytics tools, he said, “is only born from the need that an organization has.”
The COVID-19 pandemic became a proving ground for health systems and Big Tech companies to crank out advanced technology in unprecedentedly short time frames.
According to Dr. David Rhew, Microsoft’s global chief medical officer and vice president of healthcare, the focus should be on expanding tools for further use cases.
“We’re seeing efficiencies gained, we’re seeing provider experiences gained and we’re seeing patient experiences improved. So that’s become an interesting element of this, because we’re no longer talking about just trying to solve one thing,” he said. “These technologies can help us address so many other aspects.”
As an example, Rhew said Microsoft has been working in the natural language processing space to develop tools that track conversations in the exam room and bundle them into a patient’s EHR. This can reduce the amount of time doctors spend taking notes, Rhew said, as well as the burnout they may experience from juggling multiple tasks at once.
Health systems are also sowing the seeds for future innovation by partnering with colleges, in the hopes students will settle into data-driven healthcare careers.
For example, Virginia Mason and Sanford joined forces with their local universities—University of Washington Tacoma and Dakota State University, respectively—to recruit the next generation of clinical analysts, data scientists and information technology specialists straight from the classroom.
At Virginia Mason, four students from the UW Tacoma’s master’s of science in business analytics program are working at the system’s Mission Control Command Center, Metsker said. They’re tasked with creating models in the programming languages R or Python to identify ancillary service delays and make staffing recommendations.
And at Dakota State University, Nowak said business and technology students can participate in different healthcare-cybertechnology programs through Sanford’s pipeline partnership with the school. The program can attract students to Sanford before another employer catches their eye, he said.
“The quality of applicants that I get today, compared to even 10 years ago, is night and day. These are young people that want data and they want access,” Nowak said. “It’s amazing, when given that access and a little bit of free rein, what they can do.”