I interviewed Dr. Beecy for a Special issue of VentureBeat on future directions in Artificial Intelligence.

Introduction

Trestman: Hello Dr Beecy, it’s a pleasure to interview you! You have a fascinating background, including being a circuit engineer at IBM and leading a large scale risk-management initiative at Citi before becoming a practicing Cardiologist. I understand you are now Medical Director of Artificial Intelligence Operations at New York-Presbyterian, where you are responsible for governance, evaluation, and implementation of clinical AI models. 

What else can you tell me about your career? What’s brought you to this moment, heading up the journey of AI transformation at one of the world’s leading hospitals? What do you hope to achieve in this role? 

Beecy: My career has been an interesting journey, starting as acomputer engineer at IBM, where I was deeply engrained in technology and problem-solving, and later moving into risk management at Citi. In that role, I led large-scale initiativeswhich gave me a sense of how to handle complex systems,product development and deployment on a larger scale.

I recognized that I wanted to take on a more direct role in impacting people’s lives, which led me to pursue medicine. Becoming a cardiologist was a natural fit, I’ve always been passionate about healthcare and helping others, but I never let go of my interest in technology. While in financial services, Ideveloped a deep appreciation for how data and algorithms could unlock new insights. While I initially started out as a traditional cardiologist, I married my appreciation for data with the immense potential for AI to improve patient outcomes. Ultimately, that combination brought me to my current role at NewYork-Presbyterian as the Medical Director of Artificial Intelligence Operations.

My role is focused on how the enterprise can responsibly and effectively implement AI to transform care. This means overseeing the governance, evaluation, and integration of AI models in a way that not only supports clinicians but improves the quality of care for patients. AI in healthcare is incredibly promising, it also requires careful oversight. Models need to be fair, appropriate, valid, effective, and safe, especially in patient-facing settings.

What drives me every day is how this role has the potential to shape how we use AI across the entire healthcare system. My hope is that through this work, we can not only improve outcomes and efficiency, but also empower healthcare workers by giving them tools that help reduce their administrative burden. Ultimately, I hope to help create an ecosystem where AI becomes a trusted, integral part of patient care, used responsibly and ethically to enhance human capabilities.

Taking a needs-driven approach

Trestman:

You’ve emphasized a needs-driven strategy, beginning with identifying high-impact problems to be solved at the outset of a project. In your Machine Learning for Healthcare 2023 talk, you mentioned the collaborative effort involved in scoping some of your projects, such as the team of 20 clinical heart failure specialists and 10 technical faculty members iterating over the course of 3 months to define to define areas of focus and ensure alignment between concrete needs and technological capabilities. 

Looking ahead to 2025, what guidance would you offer enterprises on how to identify and prioritize AI projects that will deliver the most impact? How can organizations with finite resources balance innovation with practicality to focus on AI initiatives that are both feasible and aligned with strategic goals? What advice do you give on aligning technical teams with domain experts to facilitate this process? 

Beecy: Looking ahead to 2025, when I think about the guidance, I would offer other organizations in identifying and prioritizing AI projects, I advise to start by identifying high-impact problems that align with core business or clinical goals. The key is to bring stakeholders into the conversation early, whether that’s clinical experts, operational leaders, or end users, so you’re solving real, tangible issues with the aid of AI, not just chasing trends.

Balancing innovation with practicality is critical. I always begin with a feasibility check, ensuring the right data, technology, and resources to support the project are available. I firmly believe that small pilot projects are a smarter way to test ideas, gather insights, and it aids in avoiding big investments upfront. Once you’ have results, it is easier to scale what works. I also believe in prioritizing projects that have high potential impact and are realistic, given an organization’s capacity. With limited resources, it is important to focus where there is the most value.

For internal development, a big component is aligning technical teams with domain experts. These groups must collaborate from the start, iterating together to refine the project scope. Regular communication is crucial to bridge any gaps in understanding. I’ve seen success when cross-functional leaders help translate between the two sides, keeping the focus on shared goals.

Finally, it is imperative to define clear success metrics early on, whether that looks like improved patient outcomes, operational efficiency, or cost savings, and build a feedback loop into your process. This ensures your AI initiatives aren’t static andcontinue to evolve, providing value over time.

By focusing on high-impact, feasible projects and fostering strong collaboration, organizations can deliver AI that not only drives innovation, but also aligns with their strategic objectives.

Establishing Robust AI Governance and Monitoring 

Trestman: You have stressed the importance of monitoring AI solutions not just for model performance but for overall solution performance. Please expand on how AI leaders can ensure that they are measuring the right outcomes in the right way to capture what matters? 

In general, as enterprises increasingly rely on AI solutions, how should they approach the governance and monitoring of AI models to ensure long-term effectiveness and ethical behavior? What frameworks or processes would you recommend for organizations to manage issues like bias, fairness, and model drift over time? 

Beecy: When we discuss monitoring AI, it’s crucial to look beyond just model performance, or accuracy and precision. AI leaders need to think about the entire solution’s performance, including how it’s impacting the business, users, and even society. To ensure an organization is measuring the right outcomes, thye must start by clearly defining success metrics upfront. These metrics should tie directly to business objectives or clinical outcomes (if in healthcare, for example), but also consider unintended consequences, like whether the model is reinforcing bias or causing operational inefficiencies.

When it comes to AI governance and monitoring, I believe it is about creating a framework that ensures long-term effectiveness and responsible use of the technology. Enterprises should put processes in place to monitor both technical metrics and fairness, transparency, and accountability. One way to do this is by setting up regular audits of models to test for issues like bias or performance degradation over time. Drift is a big concern, especially in dynamic environments and monitoring should be continuous. It is also important to include domain experts in this process since they can spot issues that might not show up in technical metrics alone.

I recommend frameworks like HHS HTI-1’s FAVES, which can help ensure that models are built and deployed while beingfair, appropriate, valid, effective, and safe. These frameworks should include mechanisms for bias detection, fairness checks, and governance policies that require explainability for AI decisions. Additionally, having a robust MLOps pipeline is critical for monitoring model drift and ensuring models are continuously updated based on new data.

Finally, AI leaders need to foster a culture of ethical AI. This means ensuring that the teams building and deploying models are educated about the potential risks, biases, and ethical concerns of AI. It is not just about technical excellence, but rather using AI in a way that benefits people and aligns with organizational values. By focusing on the right metrics and ensuring strong governance, organizations can build AI solutions that are both effective and ethically sound.

Educating users for success 

Trestman: You’ve talked about how communication with end users is crucial for transparency, trust, and successful outcomes. In developing an AI tool to help predict the risk of falling in hospital patients, you found it important to communicate some of the technical aspects of the algorithm to nurses, such as the use of heavily weighted predictors. 

What advice do you have for enterprises aiming to improve the quality as well as quantity of user adoption of AI tools in 2025? How can organizations effectively communicate about the complexities of AI to users of varied backgrounds, ensuring that the technology is embraced and utilized in a way that works for everyone? 

Beecy: For enterprises aiming to improve the quality as well as quantity of user adoption of AI tools, organizations need to focus on user engagement. It’s not just about building a great tool, but making sure people understand it, trust it, and feel comfortable using it.

The way organizations communicate with users makes adifference. For example, when working with clinicians, there needs to be an explanation of the technical aspects, like why certain factors were weighted more heavily in the model, but in a way that makes sense to them. It helps users trust the tool more and understand how it could improve patient care.

One of the biggest pieces of advice I give is to tailor communication to the audience. Not everyone needs to know the deep technical details, focus on how the AI will make jobs and projects easier or lead to better outcomes. Show real-world examples of the tool in action, so users can see the impact.

Transparency is also key. Users need to know not just how the AI works, but why it makes certain decisions. Explaining these decisions in clear, simple terms builds trust. I also believe in not stopping at initial training, continuing to offer support to users with ongoing education and creating feedback loops so they feel heard and involved in improving the tool.

Lastly, it is important to think about inclusivity. Ensuring adesign and communication are accessible for everyone, even those who are not as comfortable with the technology. If organizations can do this, it is more likely to see broader adoption.

Augmenting human capabilities 

Trestman: Your talk emphasized that AI should augment human capabilities rather than replace them. In the context of your work, that means saving more lives, improving the quality of life for patients and for health-care workers as well. 

How can leaders design AI systems that effectively augment human work processes, and how should they balance the goals of augmentation of capabilities with the goals of improved efficiency and reduced costs? Do AI systems help you balance multiple goals like patient outcomes, quality of life for staff, and profitability, or do you avoid delegating that kind of decision to AI?

Beecy: In my view, AI’s role is to enhance human capabilities, not replace them, especially in healthcare where the human touch is irreplaceable. When we design AI systems to augment human work, the key is to ensure that the technology integrates seamlessly into existing workflows and genuinely adds value. For leaders, this means involving the people who will be using the system including doctors, nurses, and other key staff early in the design process. Their input ensures that AI tools are built with real-world needs in mind, whether improving patient outcomes or reducing the administrative burden on healthcare workers.

Balancing augmentation with efficiency and cost savings is anessential balance. AI can help streamline repetitive tasks, improve decision-making, and reduce errors, which leads to both better outcomes and lower costs. It is important to keep in mind that efficiency should not come at the expense of the human element, especially when patient care is involved. The goal should always be freeing up healthcare workers to focus on what they do best: caring for patients.

To balance multiple goals, AI can assist.  in predicting patient outcomes while also identifying areas where workflows can be optimized to reduce staff burnout. For now, I do not believe we should delegate big decisions like how to balance patient outcomes with profitability entirely to AI as these decisions require a level of ethical and human judgment. AI should provide the data and insights, but the final call should involve human decision-makers.

Ultimately, the most effective AI systems are those that are designed to support people, making jobs easier and improving the quality of care, without compromising the human aspects of healthcare that are so critical.

Ambient Scribe

Trestman: You’ve mentioned an ambient scribe project that aims to enable doctors to be more physically engaged with patients by freeing them from note-taking, as well as potentially help with diagnostics and decision-making. What most excites you about this project? Are there any concerns or dangers that you’re worried about getting right with this?

Beecy: What excites me the most about the ambient scribe project at NewYork-Presbyterian is the potential to transform the doctor-patient interaction. Right now, many clinicians spend so much time typing notes into a computer during patient visits that it can create a barrier between them and the patient. With an ambient scribe, we can free clinicians from that administrative burden, allowing them to be fully present with patients by listening, observing, and engaging in a way that deepens the relationship and improves care. On top of that, the technology could eventually assist with diagnostics and decision-making by providing real-time insights during a consultation. It is a huge opportunity to make healthcare more human again, while also leveraging the power of AI.

That said, there are concerns that we need to address. One of the biggest challenges is ensuring that the ambient scribe is accurate and reliable. It must capture the right information, in the right way, without causing more work for the clinician to review or correct it later. We are also always thinking through privacy and security, as patient conversations are deeply personal, and we must ensure that any data captured is stored securely and used ethically. Finally, we need to be mindful of how this technology integrates into the workflow. We want it to be a seamless assistant, not an intrusive tool that adds complexity.

Algorithmic fairness and sensitive variables 

Trestman: You mentioned that during the development of the predictive model for postpartum depression, simply excluding race as a variable—an approach known as ‘fairness through unawareness’—did not lead to the most equitable outcomes. Instead, your team opted to include sensitive demographic attributes like race explicitly in the model to better address potential biases and disparities. 

Could you expand on how incorporating sensitive features such as race into predictive models and decision-making algorithms can lead to fairer and more just outcomes? What should enterprises consider when deciding whether to include or exclude sensitive demographic variables in their AI models? 

Beecy: Through the evaluation of multiple models, we learned that simply excluding sensitive variables, what is sometimes referred to as “fairness through unawareness,” may not always be enough to achieve equitable outcomes. Even if sensitive attributes arenot explicitly included, other variables can act as proxies, and this can lead to disparities that are hidden, but still very real. In some cases, by not including sensitive variables, you may findthat a model fails to account for some of the structural and social inequities that exist in healthcare. Either way, it is critical to be transparent about how the data is being used and to put safeguards in place to avoid reinforcing harmful stereotypes or perpetuating systemic biases.

Integrating AI into patient care should come with a commitment to fairness and justice. This means regularly auditing the models, involving diverse stakeholders in the process, and making sure that the decisions made by these models are improving outcomes for everyone, not just a subset of the population. By being thoughtful and intentional about evaluation of bias, enterprises can create AI systems that are truly fairer and more just.

Fostering an AI Culture

Trestman: You have highlighted the need for ongoing education and communication, not just with technical teams but across the entire organization, to foster understanding and trust in AI technologies. What strategies can enterprises employ to cultivate a culture that is receptive to AI, encourages collaboration, and fosters continuous learning and adaptation? 

Beecy: Enterprises need to focus on a few key strategies that prioritize education, collaboration, and continuous adaptation. First, ongoing education is crucial. It is important to make AI concepts accessible to everyone in the organization, not just technical teams. This means offering tailored training sessions or workshops that break down AI in simple, understandable terms, focusing on how it impacts each team’s work and how it can improve their outcomes.

Transparency is also key. AI adoption can be met with skepticism or fear, especially when people are not clear on how decisions are being made. To build trust, leaders should be open about how AI models work, what data they use, how they should be used and the safeguards in place to ensure fairness and accuracy and encouraging questions and providing clear answers helps reduce uncertainty and fosters a sense of ownership among team members.

Cross-functional collaboration is another critical element. AI adoption should not just be the domain of data scientists or IT teams. Bringing together experts from different areas, whether that’s clinicians or operational teams, to collaborate on AI projects ensures that the tools are built with real-world needs in mind. It also helps break down silos and fosters a sense of shared responsibility for the success of AI initiatives.

Finally, continuous learning and adaptation should be built into the organizational culture. AI is a constantly evolving field, and to keep up, teams need to regularly update their knowledge and skills. Public access to commercial foundation models means the technology is democratized in a way that is different from other digital health tools. Further exposure could be through formal programs, but also by creating spaces for employees to experiment with AI safely, share what they’ve learned, and iterate on projects. Encouraging a growth mindset where people are rewarded for learning and adapting, rather than just delivering perfect results, is key.

In short, building a receptive AI culture requires education, transparency, collaboration, and continuous learning. When users across the organization understand the potential of AI and feel confident in using it, that’s when adoption and innovation really take off.

From Research to Real-World Applications 

Trestman: You mentioned the challenges in bridging the gap between what’s being published in AI research and what’s actually beingimplemented in patient-facing applications. 

What can enterprises do in general to move from AI research and proof-of-concept models to scalable, real-world applications? What are the key barriers to implementation, and how can organizations overcome them to realize the full potential of AI?

Beecy: There is a gap between what is being published and what is usable in patient-facing settings. Research tends to happen in controlled environments with ideal data, but in the real worldand especially in healthcare, there is incomplete information, complex workflows, and high stakes.

To move from proof-of-concept to something that scales, enterprises need to focus on solving real problems, not just showcasing technology. A question to start by asking includes,“what are the real-world issues we’re trying to address?” Whether it’s improving patient outcomes or streamlining workflows, the AI solution should be built around the goal. When users solve real problems, adoption follows naturally. This is similar to building a product and finding product-market fit, to ensure the solution fits an actual need and integrates well into how people work, only then can there be an expectation to scale.

Another key is building for scalability from the start. A model that works in a lab environment can struggles in real-world scenarios where data is messier, and there are larger volumes. It’s important to think about infrastructure early on, how can the system handle data at scale? How will it integrate with existing processes? And how will you keep it updated as new data comes in?

Integration is a huge part of success and AI tools should fit into existing workflows rather than disrupt them. Continuous feedback from the people who will be using these tools to make sure they align with what they’re already doing is important.

Looking ahead, adaptive AI systems could be a game changer in accelerating and sustaining the AI deployment lifecycle. An adaptive system can be built to continuously learn and adjust in real-time based on new data or changing environments. They can help refine the product fit faster by adapting to user behavior and emerging needs, which makes scaling even more efficient. Adaptive systems reduce the lag between the initial deployment and when the AI starts delivering optimal value in the real world, pushing enterprises closer to achieving true product-market fit and sustained safety in AI solutions.

So, moving from AI research to scalable solutions is really about solving real-world problems, building systems that scale, integrating with workflows, and managing the human and other dynamic aspects along the way.