Seeing the Unseen in Health Data

Q&A

Jake: Randi, I mentioned this earlier, but this our very first newsletter edition to include video…so, I guess welcome and we’ll hope this turns out great! I appreciate you so much for coming on today and allowing our audience to listen and learn from you. It’s interesting that we have worked tangentially for nearly 4 years but I feel like I've gotten to know you really just in the last year and a half.

The very first thing that I remember about you is the 2023 I2DB (Institute for Informatics, Data Science, and Biostatistics) symposium where we spent the majority of our initial conversation talking about your racing interests, the cars you own, and generally discussing your varying interests. I feel like I have to let you introduce yourself as I’m worried I’ll do a poor job summarizing you!

Randi: Well, I appreciate that. Professionally, I am trained as an epidemiologist. My skillsets and tools I use have become very important, not only because of the pandemic, but because of the focus on digital transformation in technology, processes and people.

The things that I like to focus on is how we make the science that we do translatable to the real world and truly impact health outcomes. I'm currently a professor at the Washington University in Saint Louis School of Medicine, and my colleagues and I represent the Institute for Informatics, Data Science and Biostatistics, which is all things big healthcare data. We try to harness all of the electronic health record and other data that impact health in the region and beyond. That’s what I spend most of my time doing, and then on the side, I like to race cars and talk cars.

Jake: One thing that I’m reminded of is the seriousness of your work and the impact of translational science. I’ve always taken the work I do seriously, but I never understood how much data is impacting our ability to provide quality are in this country.

Our team was just underwater that first year and focused on the technology, trying to keep up with the terminology, and ensure clarity in the data solutions necessary.  Now, I talk with so much pride about being able to support the institutions and their work. So, I’ll say thank you for this very enlightening journey.

Now, the last time that we chatted I was struck about how thoughtful you were about increasing effectiveness between data scientists and researchers! Growing that connection between the data people doing translational science and the people providing care.

Randi: Absolutely. And I also do want to thank you and Chris Lundeberg, and your colleagues at TechPartners for being mission driven with us. Having the patients and their families in mind in this work is critical. You and your team have also been patient with people like me as well. Especially in terms of waiting for me to learn the technology and terminology. Now I think that we have a much better understanding of each other, what we do, and how can support each other. It also bodes well for the communication that we have within our research teams. So, to your point about my passion for doing the right thing with data, I also want to do what's clinically relevant with data.

The communication between researchers like me and clinicians who are providing the healthcare at the point of care and beyond to patients and their families is really a critical relationship in order to create something that's clinically impactful. Ultimately what I want to see is that we create something that is impactful from a public health standpoint.  And I can give you an example of a success story. You're already familiar with this example because you've helped us with this. I've been collaborating with physicians from our Palliative Care Group as well as internal medicine and hospitalists to create a better prediction system for mortality among patients who are hospitalized. This is really important to get right, it’s imperative to know who is at risk for mortality and provide our clinical partners enough lead time to have the necessary conversations with patients and their families leading up to potential end of life and clearly documenting goals of care.

We’ve worked through the creation of this algorithm, this journey has been all about data; and like you said, we have this tsunami of data that we've had to harness even in the first 24 hours of hospitalization, which is what our algorithm is based on. There's so much data from laboratory values to vital signs to demographics, and medications. We’ve brought all of those key features together to predict 30 day mortality very accurately.

The truly exciting part about this work is now we can notify the attending physician and healthcare team that this patient may warrant a conversation about the end of life and documentation of care goals. We want them to approach their end of life with the care that they want to have. We don’t want this to be an isolating experience for patients and families, or something that they weren't counting on happening.

Quite frankly, the resources in the hospital can be better directed towards more acutely needy patients as well. If we get these patients towards the end of life in more comfortable settings and also in less expensive settings in terms of care delivery, that benefits the patients and their families. Isn’t it a gift to be able to have these conversations where patients can weigh in and have a say in how they want the rest of their life to be carried out, and not be worried about what's going to happen to their family? Especially as the patient is being wheeled away into the ICU, for example.

It's all about data and a lot of what makes it possible is technology, at least from my perspective. Sure, it’s machine learning and artificial intelligence, but it impacts real lives and it impacts them in an especially powerful way. So I'm happy to be part of it and I know that you all are too.

Jake: That is such a great example and to me, it speaks to the journey too that Chris, myself, and our team have been on. I think of our professional maturation oddly enough, initially these conversations  were uncomfortable for me to talk about. We are discussing mortality and healthcare outcomes, and we really progressed from this is “the research or health world”…can’t we just talk about about platform as a service, cluster architectures, deploying layered data APIs…but now, we want to know more, we understand the totality of the impact, we want to drive an even bigger care impact and lean in as much to your side as possible.

You just painted a perfect picture of how data is driving translational science and hitting each point in the care triangle; patient and family, providers, and research. I’ve lost three grandparents, but I’ll focus on the two that were married.

They were both aging, in fact my grandfather died close to his 90th birthday. He was living with my wife and our three kids at the time and spent the last four years of his life with us.  And his death was such a different experience being able to know what was coming, talking to him about it, discussing the plan with his physicians. It’s a big contrast to his wife, my grandmother, who we just didn't have that planning in place for as she passed nearly five years before he did. Her death was harder for all of us, the end was very challenging over her last 30 days or so.

But for him, it was “okay Doctor, what do we have to do to get him to central Missouri to go see his 94 year old brother?” We could make sure that he could do that and we had a wonderful lunch together with four generations of our family.  How much can we spoil him basically, right? Let’s make sure he’s comfortable and enjoying the heck out of himself until he can’t.

That's a very different experience and we were lucky to have that. I also recognize we were lucky to have access great medical care. I know my family and I represent the individuals that are largely already well represented in your data. You also hit on the provider and hospital side. Systems can't be everybody's everything, everywhere anymore, right? Systems are working on 1 to 2% margins and you have be thoughtful and optimize resource allocation. Finally, there's this translational science piece of you have leveraged data to find a meaningful use case and you have data that says, “we can make both of these things (patient care and system operations) work a lot better.” I know I want to continue to be in the middle of that work and moving that narrative forward with data.

Randi: Absolutely, and technology can be the enabler. When people think artificial intelligence, or hear someone tossing it with a lay audience, it makes people really nervous, especially in the context of healthcare. It makes you ask “Wait a minute, are human beings delivering my care or is the computer delivering my care?”

As you know in our case, we give the end user, the clinician, the ultimate decision about whether they or the Palliative Care Team are going to have the conversation with the patient, indicate they've already had the conversation and they just haven't documented it, and in that case it’s a good reminder. Or they are also empowered to disagree and not think it is an appropriate time to have this conversation with the patient; in other words, maybe the algorithm got it wrong.

We want to have the technology to help deliver and collate all of this information so that the clinician can make the right decision and close the loop in healthcare delivery. Otherwise it's up to the clinician to make all of these decisions based on all of the data that's available to them, physicians are very smart, they can process a lot over the course of a day, but they can't process as much as a computer can in that short amount of time.

Jake:  You have Doctors Payne, Lai, Moore, White, yourself, and many others leading translational science efforts, I’m always impressed by how everyone is thoughtful of technical integration. The entire team we’ve been lucky enough to work closely with is aligned on “AI as a Copilot.” In those conversations it is clear to me the way for  physicians to be able to operate at the top of their license, give them what they need to know, when they need to know it to be able to be the best provider that they could be.

That works all across the care spectrum. How did the team become so unified on viewing AI as a copilot? Is it really as simple as allowing technology to handle the prediction but not make the accompanying judgement in healthcare?

Randi: Yes, I think that's exactly right. This is top of mind because I was just speaking to someone this morning about this idea of just because we can do it doesn't mean we should do it. Part of this is intentionally making sure that we aren’t crowding the clinical space. We can use data and predict any number of things and we can be pretty good at predicting things utilizing medical data. The example from this morning was I could probably predict toe fungus, but that might not be the most important thing to be predicting among patients. It’s up to us to make sure that our work is clinically relevant, and it's actionable, and it's meaningful.

The reason why I advocate for pausing and assessing whether these data solutions are going to be useful in clinical practice is to make sure that we don't crowd the space. We could be letting the clinician know every second of every day that the patient is at risk for this thing or that thing or is due for this test or that test. That’s a lot of extra information that may actually detract from care.

I've worked hard with Doctors White and Moore for this palliative care project to make sure that we're delivering the right data to the right people at the right time and not overwhelming them with information. Instead, we are giving them the information that they need to enable better decision making. We're not bombarding them with extra information that isn't relevant at that time.

That’s going to be critical moving forward because we can predict a lot of things. But taking the time to ask, should we? And if so, how can we understand what's the best timing for introducing that data or analysis? When are decisions going to be made? We want to get all of this right. The right data, the right timing, and the right people to positively impact health outcomes.

Jake: One of the personal challenges I've seen with that right timing is for these pieces to work in the way you just mentioned they must be a part of the EMR. That's not easy, it is challenging to containerize a solution and make sure that you check each and every one of their process boxes. How do you keep moving through those challenges and frustrations?

Randi: I like to say, and I'm sure I've been quoted as saying, that “The technology is the easy part.” I've said that casually when I've been presenting at a conference and then the next speaker comes up to the podium and says I'm here to talk about the easy part. Then I feel really guilty. But really, none of this is as easy as flipping a switch, right? We can't just turn this stuff on at the moment that we need to have it turned on. The really the hard part, the challenging part, is the people and the processes.

I have a grant right now where we're scaling what's called a clinical decision support tool, which is a web application that's embedded in the electronic health record; it doesn't operate outside of the electronic health record. The web application is a part of the EMR, and it automatically populates with data and visualizes that data at the point of care. Getting this tool into different EMRs across the country has been the biggest challenge. It’s not because we can't do it, we've done it a dozen times. It's that we haven't necessarily done this thing with these partners at that institution. There are a lot of hurdles that have to be crossed in order to have this tool be accepted and acceptable to security teams, data governance teams, and clinical teams. We need clinical champions in place, but we also need information technology champions at those institutions.

Another one of our biggest struggles was competing demands, as this was during the pandemic. It was very evident because information technology teams were being pulled in many different directions. Many of them were being used to broker data for different stakeholders and to get minute by minute COVID estimates out of electronic health records. Individuals were being repurposed to be data people; during COVID, you had all of these competing demands and we were still trying to get our clinical trial off of the ground. We’re still trying to get this clinical decision support tool embedded in the electronic health record.

In health systems unfortunately, there's always competing demands for this very limited bandwidth of information technology resources. We need to consider that as we're moving these projects forward and really be able to move certain aspects of the process along. While we're waiting for approval by one committee, be working with another committee to push it forward. At least that is what I’ve found.

Jake: I think that's fair. For us, we're not necessarily embedded enough to see the system level challenges. We live in the operational challenges. We can be frustrated there as partners – we see a work item that may only be two or three week’s worth of work take six to twelve weeks of approvals.

I really enjoyed the conversation that we had which dove into the system level challenges of population health and data collection. Asking who's represented not only by where you get care, but who are you are, and what type of access you have to care. As we push harder into machine and deep learning and now GPTs, we're building models for the people that are already getting the best care. Those in the current rear view mirror of care have been there for some time and they're not represented well. You got pretty passionate about that.

Randi: I recognize that the data in the electronic health record is no doubt very predictive for patient outcomes. It's missing critical elements, though, regarding where the patient lives, works and plays. And when you think about what impacts health, no offense to the clinicians reading or listening, but it isn't necessarily the doctor that we have today that impacts our health, it's our upbringing, it's where we grew up, it’s the resources that we've had throughout our lifetime, it’s the health insurance that we have.

The data collected within a given health system is representative of the patients that can access that particular health system. It doesn't necessarily scale to the patients that are waiting in the wings or that are underinsured from a health insurance perspective. It speaks to and about the patients that are already being seen.

A challenge for somebody that recognizes social determinants of health is the nature of the problem. The type of health insurance we have access to by virtue of the type of job we have, the access to transportation that we have that allows us to get to our doctor's visits or our preventative care visits are true upstream problems, if you will, but they are determinants of health. Those are the factors and features that aren't necessarily represented in our most proximal electronic health record data set. It says that you have a missed appointment; It doesn't say, “oh, the bus was running late that day or it couldn't run on a day like today because of the ice storm.” EMRs don’t speak to really core features of what enables our ability to be healthy or not. We miss out understanding our patient population or our prospective patient population, if we just focus on the here and now.

I can give the example of a mom, maybe she's covered by Medicaid and she shows up today to our health system to give birth, and we've never seen her before. We have data on her birth today, but we don't have any history on her because either she didn't seek prenatal care or she didn't come to our health system for prenatal care. We just have a snapshot in time on this young woman, this young mom, and we don't know any of the predisposing factors that got her here. We don't know why her pregnancy might be high risk or perhaps it is not. We don't know that she might have a history of preeclampsia because we don't have those records in our system.

That’s a very simple example of somebody seeking care through the emergency department and us not having a holistic view. Without history of  preventive medicine or prenatal care, our ability is limited to give the best care.  We have to find out a lot about that patient during a short amount of time in order to deliver high quality care to them.

A way that we're trying to address this particular problem is to partner with other health systems in the region to get a more comprehensive picture of these patients that might show up. We want to be able to know the more comprehensive history. We might see a patient for a particular diagnosis and they could be covered at that moment with health insurance, but then they may lose their job and lose their health insurance. We might not see them again for a couple of years. The gaps in data and the gaps in health are really explainable by social determinants and less so by the quality of care that we're able to impart in any given place or time.

Jake: We all win when babies are born healthier and mothers stay healthy. Everyone in society wins, it’s always tough for me to understand, as an outsider, why these things are so hard to address. But this is going to take a lot of hard work and it’s a regional problem to solve, not just at a single institution. What do you believe will move care forward and increase equitable care in the next three to five timeframe?

Randi: One challenge with our current healthcare system is treating acute problems as they arise and not having the same incentives for keeping people healthy throughout the lifespan. We are getting really good at these drastic measures and thankfully saving lives. We should also be, in my opinion, focused on preventative medicine to keep these very drastic, heroic measures from being needed in the first place, or at least kept to a minimum. For that to happen, we need to focus on preventive medicine, primary care utilization, incentivizing wellness visits, and overall incentivizing people to stay healthy and stay engaged with the health system. Not in a sick way, but in a healthy way - again, all of this is driven by upstream factors and now we are getting into health policy.

Health policy, plays a role by creating the incentive and reward structure for healthcare delivery and medicine. In current systems, we're rewarding how well we keep people alive through heroic measures mentioned above. We aren't necessarily rewarding keeping people healthy. I'm a public health person and I know very well that it's hard to put a return on investment on something that is avoided. When we can keep the bad thing from happening, that doesn't easily translate to ROI, it’s much easier to capture expected value by replacing that knee and make sure that person is able to contribute to society again and go back to work. That is always going to be the rub with prevention - making sure that we place an emphasis on keeping people healthy even though there isn't a clear instantaneous return on investment. However, the payoff is going to be in the longer term.

Jake: This comes up a lot, fee for service and transitioning to value based care. How does it play a part in the slim operating margins within hospital systems and our then our healthcare culture? Is it going to take something like the quit smoking campaigns? Is it going to take like a full court press like that? Tough early slogan… “Take care of your body before you turn 50 please!”

Randi: Thank you for using the smoking example because I used to work for the Nicotine Dependence Center at Mayo Clinic. I can actually tie preventative care within our healthcare system and health policy from the campaigns to the quit smoking campaigns. But it wasn't just those campaigns, it wasn't just the health labels on the pack of cigarettes, it wasn't just the quit lines that different states established to help people call and get nicotine replacement therapies and counseling around helping individuals quit.

It was instead a cultural shift. It was a cultural shift because we had policies around clean indoor air. A lot of people don't remember that it really wasn't anti-smoking policies for bars and restaurants that drove improvements. It was clean indoor air policies, that delivered the right for everyone to be able to eat in an environment where you had unpolluted air. All of that collectively led to the shift that we saw, but now vaping is bringing the rates back up again. About 15 to 20% of adults in the United States are current smokers, which is a drastic change from the 1950s when all of these different targeted efforts to help curb smoking started.

For us, it’s not just a health system change that has to happen. We need policies around preventative care and incentives around preventative care to make this type of shift more accessible to everyone. Until that happens, it will remain a story in about “the haves and the have nots” Those are able to make this change are going to be able to see a difference on their bottom line when they're delivering more preventative care services in the long run. Those health systems that are integrated are going to be important to demonstrate improved healthcare outcome evidence. From there we can drive incentives so that health systems without as many resources can also make the change.

Once those policies have changed and once health insurance works differently, then the individuals are going to be empowered to start making those changes. Then we’ll see a cultural shift in our society towards being healthier. Right now, it's so hard to be healthy because it's up to you to get a gym membership and know that it’s worth it in the long run, but it shouldn’t have so much weight on the individual to make yourself healthier.  We need our system to be amenable to making everybody healthy.

Jake: One interesting trend I have seen in a few prominent places is the obesity rates across economic prosperity, that relationship of having individuals of higher means be more “fit” and have lesser rates of associated comorbidities is pretty telling in this country. But because of the longitudinal aspect of preventative health, many systems still struggle to get insurance approvals for interventions and treatments that we largely know increase the patient health outcome and decrease the long term cost of care. How do we make a big push forward to help people take better care of themselves before it's too late?

Randi: This goes back to the return on investment of prevention. A lot of the changes that we can make now among, let's say, children born at the hospital, we won't see a benefit for 40 or 50 years at least from a population perspective. We're not going to see any big shifts in how healthy that generation is until they reach middle age. Then we can compare their status to that of my generation or previous generations. If we wait another 40 or 50 years, it's too late.

Another challenge in proving this out is the healthcare delivery system isn't static, medicine isn't static. What I mean by that is we're constantly evolving in terms of how we care for patients and creating innovations that health outcomes. What we might think is attributable, let's say an improvement in health that we see among heart failure patients, well it might not be attributable to a push in society for lowering sodium intake. It might be attributable to beta blockers and the uptake of beta blockers in that population. Treatments continue to change and evolve, which makes it difficult to look back 50 years, for example, to follow that patient forward and see what happens because nothing else is standing still.

We’ll stay hard pressed to find something in medicine that we could really look back on and say, if we change that one thing, what would the counterfactual look like? What would the opposite thing that we want to see happen? What would health look like if we change that one thing.  I think that even with good historical data it will stay really difficult to do because all of the secular changes not only in our environment but also in our health systems over time.

Jake: I don't think I had thought about that! The dynamism around potential causality for health outcomes make it hard to make decisions in the future. From your perspective, how do you envision the role of technology advancing population health in the next decade? What things are you really excited about?

Randi: I’m excited about being empowered with new infrastructure to be able to come up with answers to questions faster. Unfortunately, I'm a bit discouraged in my confidence level that we will have all of the data available at our disposal to be able to make the right decisions in all contexts. We’re missing too much right now in the electronic health record, like we have talked about, and those are critical pieces of the story for all patients and those families.

One path forward that I'm encouraged by is data sharing and the data sharing that occurred during the pandemic. I can speak to what happened in our region and how even though the four health systems in our region are normally competitors, they realized they were in this pandemic together. We all realized we needed a comprehensive picture of patients and their trajectories, along what the risk factors were for having poor outcomes when a patient was diagnosed with COVID.

The health systems in our region stepped up to the plate and shared key data elements with our team and institute in order to analyze that data and inform the broader region through the pandemic task force. I'm encouraged by data sharing efforts like that because we were able to gain helpful insights that educated not only the healthcare providers themselves, but also the population about social distancing and hospital capacity and all of the other factors that were greatly impacting their lives in that window of time.

Jake: Do you think there's enough bold leaders out there in healthcare to actually do this work long term? I know the ones that we get to work with, and they’ll push to figure it out. They’ll find answers and individuals to support them no matter the area, but we need systemic boldness and leaders ready to make big leaps.

Randi: I think to a certain extent it's up to those bold leaders to bring others along. An example that I can give is implementing the clinical decision support tool that I mentioned earlier. Implementing it across cancer survivorship clinics across the US is demonstrating that it can be done, we can help not just our own but other health systems achieve innovative success. These are not academic health centers, these are community oncology practices who typically don't necessarily have the resources at their disposal that an academic health system would. Bringing them along and seeing their successes at scale – that’s the type of work that needs to be done.

It does take bold leaders to be able to disseminate the information, demystify the process for others, bring them along and show them the way for data sharing. A lot will depend on if we'll have the incentives and see the benefit of going down this road of hard innovation, but once you do, it becomes easier as you go. It's very difficult in the beginning and you alluded to a little bit of that earlier; we had to work together and learn each other's terminology and share our goals. Once we got in a rhythm, we created momentum. It is much easier to keep up that momentum and it continues to be much easier to bring people along. I'm hoping that the leaders in the field do just that.

Jake: Are you worried about care innovations being localized in academic medical centers and urban populations? You hit on your experience with community oncology centers, are you worried that path is not one everyone may follow? Could we see geographical health disparities worsen?

Randi: I think a lot about what goes on locally and the changes that we have to keep up with. What I worry about is not only keeping up with the outside world and being able to generalize our findings to other settings, but also keeping up with what's going on in our walls. It’s exemplified by the mortality model that I talked to you about earlier.

We built that first model in 2019 and then we scaled it to the hospitals within our healthcare system during the pandemic. I’m an epidemiologist, I got really worried that the model that we built in 2019 may not perform as well when people started dying of COVID, which we hadn't heard of in 2019. Even during the pandemic, our patient mix changed; we had higher risk patients very early on where the risk was high in dying of COVID. As the pandemic progressed, thankfully the illness and fatalities were less severe. So, as you know, we retrained our model internally every quarter, thankfully, we won't need to do that under normal circumstances that often. However, I think it's important to make sure that our model and anything that we put in the electronic health record maintains its internal validity to our institution and that impacts how generalizable it can ultimately be as well. If it doesn't work in our system well, it probably isn't scalable. Thinking about what we can do internally to ensure confidence in whatever intervention we deploy can help our efforts when it comes to outreach and scaling to other health systems and other populations.

Jake: Having a varied health system certainly allows for better generalizing opportunity. Much of the machine learning work can fall apart if not. There's a smart approach you’ve taken, think through the trainability of a product that you build and how does it generalize by learning across the individuals that you serve.

We’re coming up at the end of the time, what closing thoughts do you want to add?

Randi: I appreciate and am grateful for this opportunity to spout about all things epidemiological and have people pay attention. I've really enjoyed the conversation. One thing that I would like to end with is we should be thoughtful about training the next generation of data scientists, biostatisticians, informaticians, and epidemiologists to address these same concerns. Technology will make progress easier, the tools will get more accessible, but I want us to continue to be thoughtful about the work that we're doing and to ensure that the interventions that we're putting into place at the point of care and beyond are enhancing the health of everyone and not widening disparities. For that, we need the right people with the right experience to stay engaged. This goes back to the idea of understanding who gets in through the front door of care, who our patient population is, and who they aren't. Having the right people in the right places is really important to consider here. We need to be training thoughtful future leaders in this data space.

Jake: We talk a lot about the right cloud platform solutions, secure cloud enclaves, and enabling team science at scale. We’ve talked about acknowledging that the era of hero science is over. You can't do the level of research needed at the speed needed anymore with the SQL server under your desk. None of that progress will matter though if there is no one to do that hard science work.

Well, Randi, thank you as always for the amazing and insightful conversation. I know our readers, and now viewers, will enjoy it!