Real World Evidence and the Learning Health System

Richard Platt talks about the functions of three national data-collective systems, including Sentinel, PCORnet, and NIH Collaboratory. And how they used the real-world data to support doing research and making clinical decisions.

Richard Platt, Professor and Chair, Department of Population Medicine, Harvard Medical School

Director, Harvard Pilgrim Health Care Institute

“The increased complexity of health care requires a sustainable system that gets the right care to the right people when they need it, and then captures the results for improvement. The nation needs a healthcare system that learns.”

I'd like to talk with you about two related ideas. At one level they are the most straightforward and obvious ideas one could think of. And on the other hand, they are remarkably elusive and difficult targets to achieve. These two intertwined ideas are that in order to learn how to achieve the best health outcomes, we must harness the health care delivery system itself to develop evidence about what works for whom under what circumstances. And in order to do that, we need to be able to use the information that healthcare systems develop in the process of delivering care.

Although those ideas sound self-evident, I will illustrate the need from my own experience as a hospital epidemiologist, responsible for protecting patients and health care workers from acquiring infections in the hospital. The best evidence that hospitals use to guide their programs is based on a compendium that has 158 required policies and procedures developed by a large group of experts, who reviewed the existing evidence and supplemented this with their best judgement. Remarkably, the hard evidence in support of those recommendations is very thin. Over half of the recommendations are based on experts’ opinions, rather than high quality evidence. This state of affairs is true throughout medicine. Although we talk about evidence-based medicine, most of the medicine that we practice is not evidence based. The realization that is the case motivates the notion of a learning health care system -- one that develops evidence as part of the delivery of health care. I've been privileged to be part of the National Academy of Medicine team that's been working to promote the transformation to a learning health care system. The notion of real-world evidence is an integral part of this, because all or most of the evidence comes from the medical records that are developed during the delivery of care. This notion has gotten a boost recently, because the U.S. Congress made a requirement that the Food and Drug Administration incorporate real-world evidence into its regulatory approval process.

The Food and Drug Administration has noted that it has a long history of using real-world evidence. When the FDA uses real world data to understand the safety and effectiveness of medical products, it needs to assemble information from many different organizations. That is because there is no hospital, there is no health system that is large enough to address most of FDA’s questions. For the past 10 years, my colleagues and I have been working with FDA to develop a system that the agency calls the Sentinel system. My own organization, the Harvard Pilgrim Health Care Institute, is the lead organization. There are 18 health care delivery systems that participate. A number of other academic organizations provide additional expertise.

Together, we have built a data resource that has data from hundreds of millions of person years of experience, of whom 70 million are currently accruing data and there are billions of encounters. Each of them has built a data system that serves its own purposes. But in order to use the information, we need it to be in the same format. So, each of them has translated their data into a standard format. We call this the Sentinel Common Data Model. Then we spend several million dollars a year in quality checking the data. Thus, the data is extensively curated. The advantage of doing this curation is that it allows FDA to ask questions without having to stop to make the data usable for research purpose.

We do all of this through a distributed network. Having 18 of the largest health care organizations in the country participate raises many questions about the safety and ownership and privacy of the data. We address that problem by having each of these organizations keep its own data. And then we send them computer programs that can operate on their data, which is now all in the same format. And what they return to us is just the answer. How many people were exposed to a medicine of interest? How many had the outcome of interest? How many were men? How many were 72, 75, 79 years old? That's the kind of information that they are willing to share, because it helps answer an important public health question, and at the same time doesn't put them in the position of having to lose control over their own data. We put together the data from all the different organizations. In order to do this efficiently, we have built a set of tools that that can be used over and over.

I'll show you two examples of the kinds of questions FDA has asked and that we've been able to answer very quickly. The first had to do with ascertaining the kind of treatment that pregnant women receive to deal with the almost universal nausea that occurs during pregnancy. Interestingly, over the course of 15 years, ondansetron, a drug that was introduced to deal with the nausea of cancer chemotherapy has come to be extremely commonly used. There is no data about its safety in pregnancy, and yet over one in four pregnancies in the US is exposed to this product. That translates into over a million babies per year who are born after exposure in the uterus to this drug. This highlighted for the Food and Drug Administration the need to study the safety of this drug in pregnancy. It's the simplest kind of question and it's the simplest kind of use of the data. And it could only come from something that approximates a nationwide system.

A more complicated kind of question that we can answer with reusable computer programs is to do a risk-based comparison of the occurrence of complications associated with medical therapy. This is a study of the risk of venous thromboembolism, blood clots that go to the lungs. This occurrence is a well-known, although unusual, complication of oral contraceptive therapy. The FDA wanted to know whether continuous low dose treatment oral contraception had a different risk from intermittent oral contraception. In order to do that, we found hundreds of thousands of individuals who were exposed to each of those medications. We knew that they were new users of these medications because we had information showing that they hadn't received either of them for the preceding year. We could find their thromboembolic events by looking at the diagnoses that they had received. We could tell from the information that was in their records that the women who were received the two kinds of medications had different underlying risks of thromboembolism. But we could adjust for it using propensity score matching. At the end of that activity, we could tell that there was a very small difference between the risks in the two groups of individuals. Therefore, the FDA was able to conclude that there that the risk was not so large that they needed to take special regulatory action. This is the kind of study that would have taken years to do had we not had this kind of mechanism in place. The FDA uses the Sentinel System now on a regular basis.

The second example I'm going to describe is based in delivery systems, which used their electronic health records to support research about the comparative effectiveness of different kinds of medications. This is called PCORnet; the Patient Centered Outcomes Research Institute is its sponsor. Its goal is to work with patients as well as providers to identify the most important questions about health outcomes and then to address them. Like Sentinel, we have built a distributed system that has approximately 80 health care delivery organizations. Together, they have information on 66 million patients. As with Sentinel, that information is not put in a single place, but is distributed like Sentinel. There is a common data model. Here's an example of the kind of question that PCORnet was able to answer using the electronic health records. In the U.S., during the past few years, there have been three different kinds of surgical procedures used to treat extreme obesity. Sleeve gastrectomy, had become extremely popular during that time. These other two procedures had become much less popular. But there was no comparative data on the relative efficiency of these procedures. Two of the aims of this study were to ask, is there a difference in the amount of weight loss that people achieve with these kinds of surgery? Is there a difference in the weight they regain? Since many, many people with extreme obesity have diabetes, is there a difference in the improvement in their diabetes status? We were able to use information about over 65,000 surgical procedures from 41 PCORnet sites that had information on height and weight and surgery and diabetes status and other information. There was a big difference in their outcomes. Gastric banding had very little long-term improvement in weight. These other two procedures, gastric bypass and sleeve gastrectomy), had a substantial improvement in weight and much less weight regain. These are the diabetes figures. This is the amelioration of diabetes status with gastric bypass and sleeve gastrectomy doing much better than gastric banding. This evidence gives guidance to doctors and to patients about what would be appropriate kinds of care to use.

One more example of the way the health care delivery system can produce evidence through care comes from the National Institute of Health’s Health Care Systems Research Collaboratory. This National Institutes of Health program focuses on improving our ability to do actual randomized trials embedded in delivery systems. The examples that I've shown you here are ones where we make good use of information that's developed during the normal course of care. While that's good, the best evidence often comes from randomization. This is what the NIH Collaboratory intended to do, is to develop evidence about how to do trials that are embedded in delivery systems. One example is asking the question, how do you improve the adoption of well proven methods to and to encourage people to be screened for colorectal cancer? A second had to do with how you best inform people who are in nursing homes about their choices for end of life care. The third, which my colleagues and I participated in, was a study to ask what is the best way to prevent infections in hospitalized patients who are not in an ICU? This is a follow up to an earlier study we had done in ICU patients. We randomized 53 hospitals. Depending on which randomization group the hospital was in, it either continued its usual care or it adopted antimicrobial soap. We studied over 300,000 admissions. What we found was that for patients who had central venous lines, there was a significant improvement in their risk of acquiring a central line associated infection. The NIH Collaboratory puts this information together into a set of guidance about how to do research embedded in health care delivery systems. The website, www.rethinkingclinicaltrials.org, is one place to look for those.

We have learned over the years that doing research embedded in delivery system requires learning both for the researchers and for the leaders of delivery systems. We have always agreed that it's a good idea to do embedded research. But we have learned that there are five things that are really important for us to be able to do. One is look at these partnerships as long term engagements. The second, make sure that everybody is interested in the topic, not just the researcher or not just the hospital leader. No matter how interested the delivery system is in answering a question, its job is to deliver care to patients, and you can't get in the way of the operational needs. We have learned over and over that it is essential to protect the privacy of the data. Finally, it's been important to build new systems inside delivery systems to support the work that we do. We have had to spend a great deal of time and energy turning electronic health record data into data that can support research. These are the kinds of topics that I think apply not only in the US but apply in China.

We have had very robust conversations about how we might work together to learn how to do these kinds of activities together. In 2009, the National Academy of Medicine set a target that 2020, 90% of clinical decisions will reflect the best will reflect the best available evidence. 2020 is almost here. We still have a long way to go. There's plenty of opportunity for our partnership.