Implementing AI in healthcare requires strong frameworks and user trust

0

Highlights from the e‑Health Conference 2026

HN Summary

• Healthcare leaders say successful AI adoption depends on strong governance, data quality and clinician understanding of how AI tools work and their limitations. 

• Experts highlighted the importance of evaluating AI for bias, protecting patient privacy and ensuring informed consent when using technologies such as AI medical scribes. 

• New national frameworks and independent evaluations are helping healthcare organizations adopt AI safely, responsibly and with greater confidence.


A.I. is rapidly making its way into healthcare settings—but how ready are healthcare providers to adopt these tools into practice? At the e-Health Conference in Halifax this month, Christine De Maria, Physician Advisor at the Canadian Medical Protective Association (CMPA), discussed important considerations for providers as AI is rolled out more broadly across Canadian healthcare systems.

She noted that AI models often operate within “black boxes,” meaning users may not fully understand how the model reaches its conclusions. Unfortunately, AI models are only as good as the data on which they have been trained.

“Garbage in, garbage out, right?” De Maria said.

She pointed to a study published in Nature Medicine, which revealed concerning biases in large language models (LLMs). For example, the LLMs were significantly more likely to recommend invasive interventions for Black patients. In addition, LGBTQ+ individuals were approximately six to seven times more likely to be recommended for mental health assessments than clinically indicated.

“If AI recommends a discriminatory practice to a healthcare professional, the healthcare professional is still liable and could face a human rights complaint, even if it was completely unintentional,” said De Maria. “So there’s a risk that you incur if there are biases in the tool.”

She emphasized the importance of thoroughly evaluating AI tools and ensuring they benefit the specific patient population in which they are being used.

Even relatively simple tools such as AI scribes, which summarize patient-provider interactions, can raise important questions around consent, she noted. While there are currently no lawsuits involving AI scribes in Canada, De Maria said the CMPA is closely monitoring a case in California in which scribes were used without patient consent.

Even when patients provide consent, they may have questions about where their data is stored and how it is being used—and physicians need to be prepared to answer those questions.

At the end of her presentation, De Maria emphasized the need for greater AI literacy so clinicians better understand both the technology and how it should be applied. She compared AI to MRI technology—physicians do not need to understand the mechanics of how an MRI machine works, but they do need to understand the benefits, limitations and appropriate clinical uses of the test.

Fortunately, a growing number of tools and frameworks are emerging to support healthcare organizations as they adopt AI.

Notably, the federal government recently released Canada’s National Artificial Intelligence Strategy: AI for All, developed through 28 task force reports and more than 11,000 public consultations. The strategy addresses a broad range of topics, from privacy and security to ensuring Canadians have access to AI literacy training.

The Canadian Institute for Health Information has also released its Health Data Stewardship Framework to support how health data is governed, shared and used. It includes a Health Data Stewardship Implementation Toolkit, offering a self-assessment tool, glossary and inventory of resources to support sound data governance.

Throughout other presentations at the e-Health Conference, a common message emerged: organizations should take a thoughtful, deliberate approach to AI adoption through careful evaluation, staff education, AI literacy and a clear focus on the desired outcomes. AI tools are not “plug-and-play” solutions—they require strategy, purpose and ongoing evaluation.

Evaluating an AI tool

In another session, Jennifer Connolly, Co-Founder and Lead of Edge Health’s Canadian practice, discussed her firm’s independent evaluation of an AI tool used within NHS England’s urgent suspected skin cancer pathway to help reduce dermatology waitlists.

She explained that roughly nine out of 10 referrals within this pathway ultimately prove to be benign skin lesions requiring no treatment, while workforce shortages have left patients waiting longer and uncertain about their diagnosis.

The tool under evaluation, DERM—an AI medical device developed by Skin Analytics—analyzes photographs of skin lesions to identify benign cases and determine which patients require specialist follow-up, redirecting demand away from dermatologists.

NHS England’s Outpatient Recovery and Transformation Program wanted to determine whether the technology delivered on its promise in real-world practice, commissioning Edge to conduct an independent evaluation after it had been piloted in 15 hospitals across the country.

Edge’s economic analysis compared two scenarios: one in which a dermatologist reviewed DERM’s recommendations (a human-in-the-loop secondary review) and another in which DERM operated autonomously. The team calculated a cost-benefit ratio of 1.5 for the secondary-review model and as high as 2.3 for autonomous use—representing savings of up to £2.30 for every £1 invested.

Edge also completed a safety review, developing a post-market surveillance plan with clear recommendations for ongoing monitoring following implementation.

Connolly emphasized that, even in healthcare systems eager to embrace innovation, clinical AI must demonstrate its value under real-world conditions while earning the trust of clinicians and patients through regulatory approval and human validation before widespread or autonomous adoption.

Her central message was that independent, real-world evaluation generated the financial and safety evidence needed for the NHS to make informed adoption and funding decisions. That evidence ultimately helped reverse an earlier determination that there was insufficient evidence to support routine autonomous use of DERM, paving the way for its adoption as part of standard clinical practice.