Predicting HF Worsening: Speech Analysis Beats Weighing Scales

This transcript has been edited for clarity.

Ileana L. Piña, MD, MPH: Hello. I'm Ileana Piña. I'm the quality chief at Thomas Jefferson University, which is right down the street. We are in Philadelphia.

I'm lucky enough to have here my good friend, Dr William Abraham, who's a distinguished professor at The Ohio State — we know better not to say "Ohio State" — in cardiology and has been a part of many different trials. He's truly a clinical trialist.

Let me turn the page to heart failure and this definition of worsening heart failure. I always thought it was just a patient who was getting worse. If that patient gets admitted to the hospital, it truly changes their trajectory. The event rates are huge when the patients come in to the hospital and then go home. The main reason people get admitted, other than that they don't feel well, is because they're congested.

We have this term, "congestion." What does it really mean? Can we detect it ahead of time? We've been putting in monitors in the pulmonary artery system to see if we can pick up the congestion before the patient gets really sick and we end up not in the emergency department (ED) but actually inside the hospital.

Bill, what is your definition of congestion?

Congestion in Heart Failure

William T. Abraham, MD: I think the type of congestion we're talking about that leads to hospitalization for heart failure really is associated with volume retention and fluid retention.

Piña: Where is the volume?

Abraham: It occurs over days to weeks, and really the volume can be distributed everywhere. It can be in the legs and manifest as edema, but it really is the congestion in the lungs, the pulmonary edema, that most frequently leads to hospitalization because of the shortness of breath and the distress that the patients feel.

Piña: The patients' voices change. You want to hear that they're doing well. When we know our patients, you can tell that they're doing well. Break that down for me physiologically. Can we pick it up in the voice? What can we hear?

Abraham: I think clinicians can pick up worsening heart failure or congestion in the patient's voice. The most extreme example of that is at the time of their admission to the hospital. It can be very dramatic, but even if we are simply interacting with the patient by telephone, we often hear these changes. It can be a change in pitch or tone, in speech dynamics. You can sometimes hear the breathlessness of the patient.

Piña: It becomes more obvious when they're really breathless. They can't speak and breathe at the same time. That's class IV. Is there any other way that we can pick it up?

Abraham: There is. This is really the whole principle behind leveraging artificial intelligence (AI) and speech-processing technologies, with the notion that perhaps the system, the device, or the technology can hear those changes before we can. Even if we could hear them some of the time, we're not talking to our patients every day. The technology listens to the patient's speech every day and compares that to their usual, stable baseline, looking for changes or deviations that are indicative of a congested state.

Piña: The patient is actually speaking and being recorded? Is that how you do it?

Abraham: They are. This is a smartphone application that prompts the patient each morning to speak five standardized sentences.

Piña: Oh, really? Can you set the sentences or do they come standard?

Abraham: They come standardized. Within each of those sentences, there are at least 20 features of speech that are assessed, so 100 features of speech are assessed every day in each patient. Over time, tens of thousands of speech measurements are made. That allows AI algorithms to have a very high degree of fidelity in differentiating the wet from the dry state.

Piña: I'm pretty dumb about AI, but I know that you have to have a repository of data to start learning from. This is congestion we're talking about, but it could be anything. What are the datasets that exist for this?

Prior AI Studies and HearO

Abraham: The data have evolved over the past several years, first with a study done in patients hospitalized with acute, decompensated heart failure, differentiating wet from dry in the hospital, and then going on to studies in heart failure patients with hemodialysis.

Piña: That's an interesting group, the hemodialysis group. In between sessions?

Abraham: Exactly. That study was designed to begin the development of what's called, in the AI parlance, the detection engine — all of the magic of the AI that looks at this. In the current study that we just presented at the AHA meeting here in Philadelphia, we studied two groups of patients: a development group and a test or validation group. In that development group, we used a variety of conventional techniques — statistical based as well as machine learning — in order to further develop and refine this technology, and then, in the validation group, test it.

Piña: If you detect that, would you bring the patient into the hospital or into the clinic? How do you handle that?

Abraham: There are a variety of approaches, but this is certainly one of the things we need to learn more about as we go forward with next clinical trials. This would tell you with a high degree of sensitivity and a very low false-positive rate that the patient is likely to have a heart failure event within the next 3 weeks.

Piña: It's more predictive than truly anything else, but you could act on it.

Abraham: You could act on it. Absolutely. You might talk to the patient over the telephone if there are supportive data — the totality of information supports that they're on this road to congestion and decompensation. You might tell them over the phone to increase the diuretic, or perhaps it would just heighten the level of surveillance of that patient, and you might bring them into the clinic sooner rather than later for a full clinical assessment.

Piña: I have the patients still weighing themselves daily. Of course, that's my signal. If the weight is going up rapidly, that's congestion. I tend to remove the diuretics and just give it to them when they need it so that I don't stir up that renin-angiotensin system.

We often don't hear that. I'll talk to spouses and the spouses will say, "His voice is getting weaker." There you go. He just doesn't sound like he has the strength anymore, or they whisper because they can't talk.

We've been listening to these things forever, but we've never quantified them. Do you still get daily weights on your patients?

Abraham: I still get them. I think in the context of the totality of data, they're still helpful.

Piña: What do you do?

Abraham: I will tell you that studies have shown, as you well know, that the sensitivity of weight change for predicting a heart failure hospitalization is relatively low. We actually studied that in this voice-processing study as well as a comparator. The sensitivity for predicting a heart failure event, the daily weight change, was only about 35% compared with 80% for the speech algorithm. I think it's still of some value. If you see that rapid weight gain, you might increase the dose of the diuretic.

Piña: The totality of the information that you're getting from the patient. I have been very interested in the prognostic indicators, how different they are — I've heard you speak about this — from somebody who gets an oral alteration of therapy, to somebody who actually has to come in and get an IV shot, to somebody who has to go to the ED, to somebody who's really in the hospital. We have always lumped the heart failure equivalent. We started this in HF-ACTION, as you remember. Heart failure equivalents, we want to pick it up. Now we know that they're all different.

Abraham: They are different.

Piña: The prognosis is different.

Abraham: They are all important. They all have prognostic significance, but some are worse than others. You can go from that outpatient titration of oral diuretic all the way to hospitalization, and the risk goes up and up.

Piña: It's like a line going up like that. I think our new trials have got to differentiate. We know that the event rate in total is very large with this worsening heart failure, but you've got to divide it up. Somebody who gets just an oral augmentation, which they do in Europe often, is much less safe than the person who has to actually come in to the ED and get a shot even if they're sent home.

Abraham: Absolutely.

Piña: That's what the clinical trials teach us, if we delve into it.

Abraham: Yes.

Piña: This is cool. We're in the new AI space.

Abraham: Absolutely. I think it's the future. I think the good news with tools like this that are diagnostic or can help us remotely monitor patients is that they really inform the physician. They don't replace the physician. You still need the physician in the loop as the decision-maker with what to do with this information.

Piña: You need to listen to the patient. We had a session here on the art of listening, which is very important.

I want to thank you, Bill, for your time here with us today. For our audience, I hope you join us again where we want to teach our cardiologists and our non-cardiologists how to better take care of patients. Have a great day.

Ileana L. Piña, MD, MPH, is a heart failure and cardiac transplantation expert. She serves as an advisor/consultant to the FDA's Center for Devices and Radiological Health and has been a volunteer for the American Heart Association since 1982. Originally from Havana, Cuba, she is passionate about enrolling more women and minorities in clinical trials. She also enjoys cooking and taking spin classes.

Follow theheart.org | Medscape Cardiology on X (formerly known as Twitter)

Follow Medscape on Facebook, X (formerly known as Twitter), Instagram, and YouTube