Tuning Healthcare, Episode 17: Advancing the public health mission with evidence-based data

Dr. Amy Abernethy, Former Principal Deputy Commissioner and Acting Chief Information Officer for the United States Food and Drug Administration

This episode of Tuning Healthcare features Dr. Amy Abernethy, an internationally recognized oncologist, health data expert and digital health leader. Dr. Abernethy was recently the Principal Deputy Commissioner and Acting Chief Information Officer for the United States Food and Drug Administration. She also held positions of Chief Medical Officer and Chief Scientific Officer at Flatiron, as well as, Professor of Medicine at Duke University School of Medicine. Dr. Abernethy has written over 500 publications, spanning many topics, including real-world data and evidence, clinical trials, health policy, and patient-centered care.

Dr. Amy Abernethy discusses the importance of real-world data and public private partnerships in driving healthcare innovation and shares insights from her role leading the FDA through a tempestuous political environment during the Trump administration.

“I’ve always believed that if we’re really going to move the needle with respect to continuously leading the edge of learning so we take better care of people, we’re absolutely going to have to leverage data and software and novel analytics and the ability to do things in near real time.”– Dr. Amy Abernethy, Former Principal Deputy Commissioner and Acting Chief Information Officer for the United States Food and Drug Administration

In this episode, Dr. Amy Abernethy and Lumeris Senior Vice President Nigel Ohrenstein discuss: 

• Overcoming the challenges of leading an immense bureaucratic agency like the FDA
• The importance of public-private partnerships in driving healthcare innovation
• How she maintained focus and successfully drove data modernization amid the distractions surrounding the Trump administration
• The role real-world data and real-world evidence will have in shaping the future of healthcare
• The need to accelerate the transition from fee for service to a system of aligned incentives

To tune in, subscribe below:

• Read Transcript Below:

  Nigel Ohrenstein (51:18): I’m joined today by Dr. Amy Abernethy. Amy is an internationally known oncologist, health data expert, and digital health leader. Most recently, she was the principal deputy commissioner and acting chief information officer of the FDA. Prior to the FDA, Amy was the chief medical officer and chief scientific officer at Flatiron. And before that, she was a professor of medicine at Duke University School of Medicine. She has written over 500 publications, spanning many topics, including real-world data and evidence, clinical trials, health policy, and patient-centered care.

  Nigel Ohrenstein (51:55): In this episode of Tuning Healthcare, Amy and I discuss how you lead and prioritize within a massive organization like the FDA, how she drove data modernization and the importance of public private partnerships, the importance of real-world data and real-world evidence and how will it impact healthcare in the future. How you focus on your goals with the noise that surrounded the Trump administration. And if she could change one thing about healthcare, it would be to speed up the end of fee for service and to move to a system that aligns incentives. Join Amy and me as we tune healthcare.

  Nigel Ohrenstein (00:38): Amy, thank you so much for joining us today. It’s such an honor for us to have you on the podcast and a real treat for our listeners to hear your perspective. So welcome.

  Amy Abernethy (00:50): It’s a delight to be here with you. Thank you.

  Nigel Ohrenstein (00:53): So before we jump into, there’s so many topics, I feel we could cover on this podcast and we could probably talk for hours. I know when we chatted before, I think 30 minutes went by pretty rapidly, so we could probably chat for a long time on so many topics that the face the industry, and things that you’ve dealt with in your career. But before we go to there, tell us a little bit about your background. How did you end up being a physician? Tell us how you got to where you did?

  Amy Abernethy (01:27): Interesting! Well, so I started off thinking I was going to be a basic scientist, and when I was working in the lab building melanoma vaccines, so these were vaccines intended to stimulate the immune system and create a response against skin cancer. Yes, I liked the laboratory part, the mice and everything else, but I was most fascinated when I would end up in the clinic, giving people the vaccines on clinical trial, and then also asking questions, like, “Why did you travel all the way to Durham, North Carolina, from Ohio to get this vaccine?” And I would learn about family history, motivation around going to the granddaughter’s wedding, all these aspects of real life that health and healthcare is really all about. And so, I quickly realized that I wanted to transition my time from the bench science to the clinical side, and the clinical science.

  Amy Abernethy (02:26): After becoming an oncologist, I really focused my time and energy on trying to figure out how do I take better care of the person sitting in front of me? So if you think of me as a melanoma doctor who’s taking care of people with skin cancer, how do I take better care of people with melanoma, so that the care of the next person is a culmination of all of the best knowledge that came before her, and her care gets reinvested into the overall system of care? And honestly, that’s been the problem I’ve been trying to solve my entire career. I ran the Center for Learning Health Care at Duke, where we were trying to do this from the academic side.

  Amy Abernethy (03:04): I then went to a health tech startup, Flatiron Health. We were trying to figure out how do we motivate Silicon Valley thinking and venture capital to try and solve this problem, including the organization of cancer data. And then ultimately, I went to FDA, where I was trying to think about how do we help make sure that the regulatory apparatus is ready to scale as we have more and more treatments coming down the pipeline. And I just left the FDA about a month ago, after serving for a couple of years as the Principal Deputy Commissioner and the acting Chief Information Officer, really focused on making sure that the regulatory apparatus is ready to go along with the rapid advances we have on the discovery side and the possibilities as it goes back to this kind of core question of taking better care of the person sitting in front of me.

  Nigel Ohrenstein (03:52): And it’s fascinating, and as you’ve clearly gone through your career, the impact that you can and have on people is obviously clearly growing. Tell us a little bit about the transition from academia to a health tech startup, and then from the private sector to government?

  Amy Abernethy (04:15): Holy cow, right? Three massively different careers, huh? I think it’s easier to make those really different career moves when you have a raison d’etre, right? Like, you’re trying to solve one basic problem, and the question is, how do you bring the best of what’s possible in a particular career setting, to solving that problem? And so I would say that, it may seem like I’ve had three really, really different careers, and I’m about to do another one, so next chapter coming, but the reality is that I have been constantly trying to figure out, how do I make sure that I’m bringing my best self as a physician, a physician scientist, a leader, a health tech person, to this particular place where I’m working?

  Amy Abernethy (05:07): And what I have found is that kind of focus on vision, whether it is at a health tech company, standing up in front of the company and sort of saying, “Here’s the North Star of what we’re trying to,” do or being at FDA and having 18,000 people need to transition to teleworking overnight because of the pandemic, being able to have each of those organizations sort of understand the why, and then doing my best to bring my part of solving for that why, within the context of those organizations, has been how I get my job done.

  Amy Abernethy (05:43): I would also say that what’s been interesting having had all these different jobs is that it allows for what I would call, “Intersectional thinking.” The ability to see what’s possible by knowing health tech, and knowing government, and understanding how things get done on a governmental and policy frame, now sort of thinking about, what’s that intersection look like, and how are we going to do it differently in the future? And I think that’s been one of the great values of doing all these different careers.

  Nigel Ohrenstein (06:13): So I couldn’t agree with you more. I think we often make the mistake of trying to find people that have had exactly the same experience to put them in the next role, and we don’t benefit from that, we benefit from people who bring different experiences, and that ultimately makes us think differently, and drives innovation and change. Let’s delve a little bit into intersectional thinking, because that actually that’s a lot about how Lumeris was created. We wrote a white paper a decade or so ago more than that now on what does it mean to be a collaborative pair, and really create that intersection of pair and provider, that true collaboration. And so, as you think about what you’ve achieved at the FDA, what would you say you pulled the most from your sort of previous experiences, that sort of framed how you approach that role?

  Amy Abernethy (07:12): Well, so first I might highlight why when I was at FDA, I also took on the role of CIO.

  Nigel Ohrenstein (07:22): As if you needed more work to do, because if it wasn’t enough?

  Amy Abernethy (07:27): Right, just in case I needed more work. What happened was, and I thought about going to FDA, and I enumerated the things that I was going to make my priorities to work on, I said three things: patient centricity, real-world data and real-world evidence, and accelerating clinical evidence generation. Those are the three things I thought I could focus my time and energy on, and that would ultimately lead to an overarching goal, which was around improving precision healthcare.

  Amy Abernethy (08:03): However, I get to FDA and I was like, “Oh, wait a second. So if FDA is going to work with real world data and real world evidence, there don’t have to have a way of ingesting data and analyzing it. And also, holy cow, if we have more and more precision medicines, and we’re more personalization based on understanding what real people want, there’s going to be more drugs to be evaluated by regulators and more work to do, and so we need the FDA to be ready to scale along with all of these new capabilities.” And so that’s the reason I took on the role as CIO, and sort of advanced the data and tech modernization at the FDA.

  Amy Abernethy (08:49): Now going back to your question, which was about intersectional thinking. First of all, I think that the opportunities learned in my prior roles, whether that was, how do you use data and technology and product mindset to think about building scalable solutions, right? That was one of the things that I had learned. Another one in my Duke role was: how do I align up clinical trials and observational research and how do I leverage data in new ways? And so, I was able to kind of bring those to what I was seeing at FDA, and quickly be able to pinpoint a problem that I thought was indeed a problem to solve. So that was one part of the intersectional thinking.

  Amy Abernethy (09:31): And the other part was something completely different, which is that I had learned at my prior roles that in order to solve hard problems, especially at the interface between science, and software, and an analysis, and analytics, and modern thinking, actually have to have new ways of getting all of those diverse and usually non-intersecting disciplines to learn how to work together and talk to each other. And so, you needed to figure out how do you create equal and equalized power structures, so that everybody can sit at the same table and listen to each other? You have to figure out what’s lingua franca? What’s the common language that allows everybody to talk in the same frame? And so I think that what I had learned in my prior roles that then showed up at FDA was not only, kind of questions as it related to data and tech, but actually also questions as it related to helping all of the different disciplines work together in an even and collaborative way, as well as talk to each other in a way that everybody could not only listen, but also understand.

  Nigel Ohrenstein (10:46): Let’s take a step back for a second, because everybody knows the FDA, right? And it’s got such a massive role in, in protecting public health, but I’m not sure everyone always appreciates the true scale of the responsibilities of the FDA. So take a moment, just frame for us the gamut of those responsibilities that you were thinking about, almost every day, for the time you were there?

  Amy Abernethy (11:19): FDA is a science-based public health agency. The responsibility of FDA is to protect and promote public health, and also to promote innovation, which I think is an important part of FDA’s remit. And when you think about this, FDA’s remit is massive, as you just described, but in ways, sometimes people don’t realize. First of all, FDA regulates probably between 20 and 25% of the world’s economy, right? Because, as sort of the world leader on the regulatory stage, that also is not just us, but it’s actually world economic implications. And this is not just food and drugs, but also animal products, cosmetics, dietary supplements, tobacco, medical devices, and you get my point; the number of different product types or product categories, the different regulatory requirements that sort of impacted each of these, and how you make sure that any particular product meets the accepted regulatory standard for that product, and now could get into the hands of people writ large, is the responsibility of a regulatory agency. A regulatory agency that is too conservative and too scared means that innovation doesn’t happen, and a regulatory agency that’s too porous, and too easygoing, means that we have potential catastrophes. And so getting that right is particularly important.

  Nigel Ohrenstein (12:50): And funny you should say about the breadth, because I moved locations in my home between my last call and this call and as I was walking downstairs, I was trying to think of everything in my home that is regulated by the FDA. And it’s probably more than 25% of what’s in my house, that’s for sure. It might be 25% of the world economy, but it’s more than 25% of what’s in my house is probably regulated by the FDA. It’s mind boggling.

  Amy Abernethy (13:16): It Really is crazy.

  Nigel Ohrenstein (13:18): I don’t have some of the things you regulate in here, that’s for sure. So, tell us a little bit about when you have such a broad remit, how do you prioritize? One of the mantras that we always operate by is, if everything’s equally important, everything’s equally unimportant, right? To sort of to drive people to prioritize, because if you have 10 things on your list, you obviously had way more than 10, right, and everything’s equally important then you’re basically not saying anything is more important. So how do you prioritize when you have such a massive thing? And that would be sort of one. And then the other aspect too, then how do you hit that balance, right? Which is so hard because the conservative person’s going to say, “I don’t want to cause damage”, right? But the innovator is saying, “I need to push the ball forward”, which is clearly an amazing part of your mindset is I need to innovate and push the ball forward. It’s hard to balance all those things and come up with a list of priorities.

  Amy Abernethy (14:28): I’m going to first go with the how you prioritize. And as you were asking the question, honestly Nigel I was like, prioritize what? How do you prioritize what to learn when the remit is so large and there’s like 20 plus books with the regulation? You can’t possibly know all of that, right? You got to figure out what are you going to prioritize to learn or how are you going to learn it? There’s what are you prioritize to do? So if I’m going to have some priority activities that I’m going to try and move the needle on differently over a defined period of time, how am I going to decide what’s important? And then also what are you going to prioritize to motivate? There’s 18,000 plus people at FDA, how do I prioritize the motivation across that? That was what I was first thinking and trying to figure out when you were asking the prioritization question and let me just hit on those three things. So with respect to prioritization on how do you prioritize the learn, I couldn’t possibly learn all the regulations.

  Nigel Ohrenstein (15:32): Oh, I’m sure you could, if given enough time.

  Amy Abernethy (15:36): Maybe given enough time. But I always find people are asking me to rattle off some regulatory thing with some number of blah, blah, blah. I’m like, I don’t know, I’ll Google it and figure it out. And instead what I found I needed to do was prioritize frameworks. Like what would be the approach regulators would take to this particular problem and what would be kind of the overarching framework? So a risk-based framework, for example, and understanding that if we are going to put FDA resources towards it, there’s a couple of different ways we can apply resources. So what’s the sort of set of resources that we’ve got and the framework within which we’re going to do that.

  Amy Abernethy (16:21): And then outside of that and understanding how do I quickly find the details for any particular scenario as I need them. And it was interesting, one of the things I was responsible for was advancing FDA’s work as it related to CBD, cannabidiol. And this was a situation where figuring out how FDA was going to regulate in this sort of rapidly advancing space. I kind of got brought into this and the number of product categories that CBD is relevant to is massive, right? Like CBD and products that think of themselves as dietary supplements, CBD and food, CBD as a medical drug, right? We even see CBD in devices. And so now starting to understand how a particular product shows up in these different categories, helped me think.

  Amy Abernethy (17:14): And then thinking about what are the things in FDA’s toolbox like the regulatory decision-making, deciding when it’s something can go on the market. So regulatory decisions in that category versus enforcement and when you take things off of the market. And so like thinking about that. So framework, what are the things in the toolbox the FDA has to apply? And then how am I going to quickly find the detailed information when I need it to understand, for example, what does a dietary supplement really mean and what are the rules as it relates to dietary supplements? So that’s kind of the first thing around privatization was I had to come up with a schematic and that was my schematic at least. That make sense so far?

  Nigel Ohrenstein (17:52): Yes, it makes sense.

  Amy Abernethy (17:54): The second thing you asked about privatization was what am I going to spend my time on? And that I think comes back to raison d’etre, right? I’m going to choose how to focus my time on things where I think that I can move the needle, but line up with my big why, right? Going back to how do I take better care of the person sitting in front of me in the clinic? And I looked across FDA and there was a ton of different things that are important to work on, right? So mother and infant nutrition is an example of something that I thought was really, really important to work on. Important, but when I thought about making big impact on what I thought I could do best within the context of my raison d’etre, probably didn’t line up perfectly, if that makes sense. And so what I tried to do was say, here are the things that I think I can do and bring to the table that are going to move the needle on those things that are most aligned with my big why. And that’s how I ended up working on data and tech modernization, evidence generation, real-world data, precision medicine and precision health care. That’s kind of how I did that and then what I did in those categories.

  Amy Abernethy (19:05): And then the last part is prioritization as to how you motivate other people. So your big why or my big why, that’s going to be where at night and on weekends, I’m spending my extra time, but also there’s others across the organization that are going to want to put their energies into important areas. And I sort of thought a lot about how do you motivate others to sort of work in line with their big why? And so that’s where for example, maternal and infant nutrition comes in. I’m going to do what I can to help many people see this as an important problem, because others then can figure out where does that fit in their raison d’etre and start to move the needle on that one. And so that gives the opportunity for me to distinguish my time as prioritization and the work that I’m pushing forward, as opposed to now other things that I think are important but didn’t necessarily hit my top list because it wasn’t as perfectly in line with my skillset. And so that’s kind of how I distinguish between the two.

  Nigel Ohrenstein (20:54): So Amy, I think you’ve just shared one of the sort of most fundamental leadership lessons which is how do you enable the team that works with you to have sort of the same passion to get up every day and drive that agenda that you might have. And the thing that I think, I frame it slightly differently as sort of how do you maintain that founder’s mentality within a business, right? So the first 10 people are close enough to the founder so they understand the mission and they’re driven and they joined because of that mission. And how’d you drive that to the next 10 and the next 10 and the next 10. So everybody comes to work with that founder’s mentality.

  Nigel Ohrenstein (21:39): I think, in fact, you’ve just shared how you do that in a government agency, which is truly mind boggling, right? Because to do that in a company that at least from my mind seems easier than how to do it in a government agency. And I think you perhaps shared perhaps a really fundamental leadership lesson. So thanks for that!

  Amy Abernethy (21:58): Thank you.

  Nigel Ohrenstein (22:00): Let’s delve a little bit into some of those areas you focused on. Let’s start with data and tech, because that’s something that you were passionate about before you came to the FDA, as you said, it was one of those driving things. You started initiatives before the pandemic. Tell us a little bit about how you’ve innovated around data and tech and the initiatives you’ve driven during your time.

  Amy Abernethy (22:30): Absolutely. First of all, for full disclosure, I am a data person. So I would say that kind of if you think about 10X skills or 10X attributes, data would be my 10X attribute and I am not a software engineer or a software person. I had my first job when I was 16 at NASA and really sort of in programming and thought I was going to become a computer scientist before I thought I was going to become a basic scientist. And that all is only relevant to say that I am intrigued by the possible and celebrate engineering and software design. But really this is not my core thing that I understand or know.

  Amy Abernethy (23:19): But at the same time, I’ve always believed that if we’re really going to move the needle with respect to continuously leading the edge of learning so we take better care of people, we’re absolutely going to have to leverage data and software and novel analytics and the ability to do things in near real time. And so that was the backdrop. When I got to FDA, as I mentioned, I could see that we were going to need to leverage data and technology differently in order to help the FDA scale and help the FDA be ready to ingest data differently. And the other thing that I could see, and I think this is really important, is that the way that the FDA uses data not only ingests it, but makes sense of it, puts it to work, also influences how all the industries regulated by FDA use data.

  Amy Abernethy (24:19): If I could start to change the way that FDA thinks about what’s possible with data, what does good look like, then I could see how that would have an important ripple-on effect. The other thing is that I was involved in high-tech. I have been involved in a number of large scale data and tech initiatives in healthcare across time, and I think that many of them have been hamstrung by what I call vendor think, which is we’re healthcare and we know what’s needed and so we’re going to write a number of specifications, then hand them over the fence to the data and tech industry so that they can build to our specifications. And then, I’m not calling this waterfall versus agile design, it’s actually more about envisioning the future and how it ultimately gets built, ideally gets built in an agile way, but what I really mean is that what’s historically happened in healthcare, is healthcare envisions the future and then hands that vision over to tech and as a result, the possible is often missing from the vision.

  Amy Abernethy (25:28): And the other thing that I felt was really important was that we find new ways of bringing software expertise and tech and regulatory and healthcare expertise together so that we could envision the future together and now think about building towards that possible. And that ultimately led to both our data and technical modernization action plans at FDA, which really were about how do we build the right technical foundation? How do we have the right people? What’s the right budgeting and governance models? And then what are we going to do to showcase what it looks like when we build technology in the right way, when we put data to work, when we have conversations that are collaborative with the data and tech industry to a vision, envision a different future, and then start to move our way in that direction?

  Nigel Ohrenstein (26:19): It’s incredible because I just know trying to get things done in private companies is often difficult. Trying to implement these changes in a government agency must be a monumental feat. And when you focused on these initiatives, how do you set goals? How do you set goals that are achievable in sort of a public government environment? Because people obviously aren’t bonused, it’s not like a private company where we hit these goals, you’re going to get a bonus and so everyone’s then driven to go the extra mile. How do you set goals to achieve some of the things you wanted to achieve around, stay focused on sort of the data modernization?

  Amy Abernethy (27:16): Interesting question. It was actually worse than you just described because not only it was sort of like no financial incentives if you win, there were public health incentives, and that’s really important by the way. People who work at FDA are incredibly motivated by the public health mission. So that actually helps, but there’s no financial incentives that you can build against and worse, the budgeting cycles are 18 to 24 months out. Try setting goals when you actually can’t ask for new budget and know you’re going to get it for two years, right? And I was only there for a little over two years. So in order to get there, I needed to convince both the agency at large, as well as the data and technical organization of what was possible, and then move from that direction. A lot of like how we got there and how we set goals is fairly tactical, but I think is really fundamental. The first thing that I tried to do was set up a list of expected objectives and those objectives included focus on our people. Right? Make sure that the people who already work, especially in the data and technical organizations, know that they’re important and that we find new ways to bring their voice to the table. That was really important because what I found was that our hierarchical organization not only was creating sort of a fairly downtrodden culture, but also that there was no way of finding dollars available or taking waste out of the system, because there was nobody to speak up to say, “We might do X differently.” Focus on people also then allowed us to identify untapped budget.

  Amy Abernethy (29:13): The second thing then was we put in place a budgetary initiative that we called Project Blue Sky, where we identified 10% of our overall budget. In this case, our overall budget was $325 million. We identified 10% that we were going to redeploy. This time, the commitment was, “We’re going to redeploy it on ourselves to make sure that we can do our job better.” Then, that went to the third part, which was in doing our job better we set a series of operational excellence goals so that we could see if we helped our people have more of a voice. If we redeployed our own dollars in a way that allows us to do our job to get better, then we could actually see what progress looks like against a set of true operational roadmaps.

  Amy Abernethy (29:57): What’s interesting is, that sort of basic strategy led to two things. First of all, for once people felt like there was a real commitment to them that then allowed everybody to stand taller, have more voice, and then actually do their work better in a way that then the whole agency took notice. The year before there had been no way we would have been able to take 18,000 people to full teleworking in 24 hours. Yet, we did it seamlessly and in a way that the whole agency celebrated. Again, with no new budget. That kind of ability then allowed the next thing, which was now we could ask for new governance models for the whole agency that said we need to make sure that the data and tech organization has a voice at the highest levels of the agency.

  Amy Abernethy (30:48): The CIO needs to report to the principal deputy commissioner. It was easy because I happen to be the principal deputy commissioner. I was able to act in that way until we made that happen, but we were only able to truly action it because of the fact that we’d showcased what was possible by getting people to work in new ways. Then, that allowed us to now set a new set of roadmaps and go to congress and say, “Okay, here’s the new money that we need.” It’s interesting to me looking back how basic it sounds and how truly revolutionary it was in getting things done, especially in a world where almost always the only way to get things done is to ask for new budget authority. That comes with all the political challenges that go along with getting new budgetary authority.

  Nigel Ohrenstein (31:40): Touching on the political challenges, when you delve into what has been achieved like at the FDA, at CMS, HHS in general, right, some amazing things were done during the administration, but obviously there’s always a lot of noise going on as well, just by the nature of the administration. Did you just focus on your job and the noise was just off to the side? How did the whole sort of aura that went around the administration impact you from a sort of day to day perspective?

  Amy Abernethy (32:19): How could you just focus on your job and ignore it and look at your toes with all that noise going on? The answer is no, but at the same time, how do I lead? As a civil servant at FDA, how do I lead an extraordinary group of 18,000 plus career civil servants, scientists, and physicians, and experts of all kinds, that clearly can’t miss the fact that the president is tweeting at us. Right? I thought a lot about this, honestly. The first is that one of the things that I learned at the agency, and Jeff Sherman, who is the director of the devices center CDRH was really the person who taught me this, is that the agency leadership, part of our responsibility is to act as the insulation, the sponge in between the political noise and the rest of the agency so the agency can really focus on getting the core work done.

  Amy Abernethy (33:29): That was sort of one thing that was really important is to acknowledge that our job was to actually act as that cushion. Sometimes, that means you take a bit more of the hit, but that’s kind of the way it works. The second thing is like, at least from my perspective, it was nonsense to act like it wasn’t going on. People are human beings and they read the newspaper and their grandmother calls and asks about it. By the way, it’s the conversation at the dinner table. If you act like it didn’t exist, you’re actually not giving enough respect to the people for whom you’re responsible. Tried to address it head on in a variety of ways, but through a variety of town halls and team meetings and helping some supervisors understand what to say.

  Amy Abernethy (34:26): That was hard because, practically speaking, you have in government, you have a lot of rules of what you are and you’re not allowed to talk about, especially in the prior administration. I think that we had to equip ourselves with ways of having a safe conversation, but the ways that you can have a safe conversation is to say like, “Let’s not put our head in the sand, let’s acknowledge the things are hard, but in a time of a public health emergency, this is our responsibility, public health.” And like really focus on what’s important.

  Amy Abernethy (34:59): The last thing I would say, and this was really important to me personally, and this is not about the politics. FDA doubled our workload. We had a kind of core fundamental responsibility in the middle of the pandemic, but as I said before, we’re human beings. Right? The things that are hard and that have hit the mental health of every American hits the mental health of the agency as well. We very quickly put in place the kinds of like simple things, like focus on your family, making sure you get enough exercise. Here’s what mental health and counseling opportunities look like we actually needed to surface? How do we talk about this? How do we help supervisors know how to have safe conversations with employees? Because we are all in this together and burnout is a challenge. The sense of sometimes feeling that the world is against you in the middle of all those politics is a challenge, et cetera. Finding ways to acknowledge that it’s hard and give people the equipment and tools that they need to solve through it, I think was equally as important.

  Nigel Ohrenstein (36:14): Very powerful. I’d love to spend a few minutes on real-world data. I think it’s a fascinating topic that’s really, through the pandemic, I think come much more to the forefront of people’s minds, because prior to that, I think it was definitely a topic that really focused on those people like yourself, who are scientists and just sort of thinking about how do I match real world data with sort of either clinical trial or evidence or whatever else we have. I think the pandemic brought that to the forefront because you saw treatments coming quickly and saying, “Well, that treatment’s only been tested on these people, but how can I pull real life data and apply it to other people?” Or obviously we forever only had vaccine trials on a subset of the population, but because the vaccine was so much more in the public forefront, everyone was now focused on, well, who are the 20,000 people that had the vaccine trial? How does that work? Clearly, you’re not going to ever cover everybody who has every potential co-morbidity or whatever else in a clinical trial.

  Nigel Ohrenstein (37:26): Tell us a little bit about your perspective on real-world data, where do you see it going, and how do you sort of involve the mindset of people that are sort of, what I would call, for want of a better word, traditionalists who believe this is what we’ve observed, and therefore we can only do what we’ve observed.

  Amy Abernethy (37:51): Let me kind of step back for a second, just do a couple of definitional things to make sure that we’re all on the same page, that this landscape of real-world data and real-world evidence, we used to call it outcomes research, or comparative effectiveness research, and then it went through a resurgence as patient centered outcomes research. Now these days we call it real-world data and real-world evidence, but it’s really an expansion on the conceptual model of data generated outside of a traditional clinical trial setting. Now, sometimes it’s prospective. For example, a prospective registry or a pragmatic clinical trial, a lot of times in 2021 and beyond, we’re also talking about data that’s passively collected through a variety of different means. Electronic health record data, administrative claims data, the accelerometer in your watch, like these are all different types of real-world data sets.

  Amy Abernethy (38:43): These days, we can also check a Reddit and Twitter, like these are new types of real-world data. Then, the question becomes, how do we make sense of it in a way that provides adequate and compelling information that can be used to make a specific decision or provide specific observations or characterizations? If that decision is a regulatory decision, and then we sort of ask this question, how might we use real-world data and real-world evidence for regulatory decisions? Certainly, 21st Century Cures as a landmark mark piece of legislation has included the expectation that the FDA will provide guidance on how FDA might consider using real-world data and real-world evidence. Specifically, in two use cases, expansion of a label, so for example, a new indication for a drug that’s already approved or post-marketing commitment or post-marketing requirements. Looking for example at safety evaluation of a drug after it’s been put on market.

  Amy Abernethy (39:50): I think that in the FDA context, actually really the core maturation of real-world data and real-world evidence has happened largely in the device context where the evaluation of devices such as continuous glucose monitors can happen through real-world data that allows you to understand how these devices perform across time. I can tell you that the real-world data, real-world evidence space was already hot and heavy, and people are arguing whether or not this was an appropriate direction to go and should real-world data and real-world evidence have an equal stature to a traditional clinical trial, or what might that look like?

  Amy Abernethy (40:31): I would argue that up until the pandemic, the book of thinking was largely what I call replacement product for clinical trials, like how can real-world data and real-world evidence serve as a replacement product for clinical trials so that clinical evidence generation can be faster or less expensive or not have to randomize patients and all these things. I think we learned a ton about what was real and what wasn’t in COVID. First of all, you highlighted vaccines. We’ve certainly seen from Israeli data as an example, that real-world data and real-world evidence can help us understand medical products in context and give us an understanding of effectiveness at a population scale once products have been approved or authorized.

  Amy Abernethy (41:17): I think that’s really where this goes and vaccine example from Israel, and even some of the work done in the United States really is like, this is what it should look like. There’s a ton of other things that we learned in COVID. One of the things that we learned in COVID is to expand our thinking about data. You notice that I keep calling it real-world data and real-world evidence, like the data is the important part here, not the evidence. Really, sort of thinking largely about how do we think about data sets of many different types? How do we think about linking them together? What happens when you put genomic information together with the EHR data? Blah, blah, blah. I think that there’s been a huge expansion of our understanding of how datasets can be put together in new ways, but also the kind of critical issues, such as data curation, data characterization, data cleaning, all these things.

  Amy Abernethy (42:08): That actually brings me to the next thing I think we learned in spades during the pandemic, which is that there’s a lot of people talking about it, but not that many people who really know how to do this in the best way. And we’ve seen a number of shenanigans during the pandemic, whether this was the Surgisphere episodes, where these two papers had to be retracted from The New England Journal, and The Lancet, or just ton of investigators who published some paper from their health system’s electronic data warehouse, and called that now the answer to every question anybody ever wanted to answer.

  Amy Abernethy (42:46): And so that then highlights, I think, the next thing that we really learned in COVID is that there’s a lot of work that needs to be done in the real-world data and real-world methods space, and there’s just a ton of methods work still to be done. And that methods work isn’t going to stop because the datasets are probably going to keep getting more sophisticated and complex, and the methods are going to need to keep going.

  Amy Abernethy (43:09): The last thing I would say is that we’ve learned a lot about the possible, though. And one thing we learned in the context of the possible is that I think we’re going to move away from calling it real-world data to just data. By the way, we can use data, often from real-world sources, to better manage supply chains and real world performance of diagnostic tests and figuring out how to move ventilators around the country and figuring out what are the contours of the pandemic, and what’s the risk of developing long COVID syndrome, and all these other things.

  Amy Abernethy (43:46): And then if we’re going to have all these new ways of using data, the regulators need to learn how to use it too. So there’s this thing I call regulatory muscle, which is having regulators, just like the rest of us, understand the possible, so that there is the ability to discriminate what does good look like. And that goes back to my core point around data and tech monetization at the agency, which is that the more that the regulators can put data to work at the agency in new ways and teach all the industries regulated by FDA to use data in new ways, we’re going to start to see this proliferate across.

  Nigel Ohrenstein (44:22): That’s great. Thank you. We could probably go on talking for a while, and we like to end with what we call the quick-fire round. But before I ask that, I have to ask one other thing. You threw in there earlier that you worked at NASA at 16. I think at 16, I only knew where my next soccer game was coming. How does someone end up at NASA at 16?

  Amy Abernethy (44:47): So at 14 and 15, I had gone to the talent identification program at Duke. This was when it had first started. It was the first and second year, and I started computer science and biology. And then I had this opportunity to go to NASA for the summer. Anyway, long story short is I really wanted to study green sea turtles. And I was going to go to Cape Canaveral and study green sea turtles, but they didn’t need anybody to do green sea turtles. And they’re like, “You actually are pretty proficient with programming. We have this need in the AI lab, artificial intelligence.” And interestingly, I was responsible for programming this robot to do this experiment at Kennedy Space Center. And my first publication was when I was 16, about programming this robot in the artificial intelligence lab at Kennedy Space Center. So there you go!

  Nigel Ohrenstein (45:48): Wow -amazing. That’s incredible. So, we do like to end with what we call the quick-fire round. What’s the best piece of advice you’ve ever given?

  Amy Abernethy (45:54): The best piece of advice I’ve ever been given was from one of my mentors who told me you need to enumerate the three principles, and you can only have two to three, that are going to guide your life. And then every time you have a tough decision, go back to your core principles and decide what are your core principles telling you about how to navigate that decision. And I have literally used that at every single tough juncture.

  Nigel Ohrenstein (46:18): That’s good advice. What do you do to relax, to have fun when you’re not changing the US healthcare landscape?

  Amy Abernethy (46:26): I love to exercise and just be outside. So I, you probably guessed, grew up in Florida. I am definitely into the heat, so I will be outside all day long and find different ways to walk around, et cetera.

  Nigel Ohrenstein (46:42): If you were starting your career again now, what advice would you give to your younger self?

  Amy Abernethy (46:49): Oh, wow. I think that the advice I’d give to my younger self is the advice I wish I’d had, but I ended up ultimately taking, which is don’t just assume that you need to follow the path everybody else has taken. Follow the path that is line with your passion.

  Nigel Ohrenstein (47:11): And then finally, if you could change one thing, I know that’s a hard one, one thing about healthcare, what would it be?

  Amy Abernethy (47:21): I would probably change the incentives of the payments and healthcare delivery. I think that unfortunately the fee for service payment systems, unfortunately drive a lot of bad decisions.

  Nigel Ohrenstein (47:40): And we obviously couldn’t agree more because that’s the mission that we get up every day to follow.

  Nigel Ohrenstein (47:46): Amy, it’s been an absolute delight. We could probably chat for hours, and there’s so many other topics that we could have hit on that we didn’t but thank you for your time. Thank you for your insights. Thank you for what you’ve already given to the country. I can’t wait to hear what the next episode is. I have no doubt that you will continue to make a massive impact that improves healthcare, not just for the people that you serve, but for all of us. So thank you so much!

  Amy Abernethy (48:15): Thank you -it was fun.

  Nigel Ohrenstein (52:37): Thank you for joining us today. Please follow us on your favorite streamer. And don’t forget to rate us, as it helps others find our podcast. I hope you continue to stay healthy, and please join us next time as we tune healthcare. This is Nigel Ohrenstein in New York.