Artificial intelligence (AI) is being increasingly integrated into healthcare delivery, from clinical decision support tools to administrative tasks. While these technologies offer opportunities to improve efficiency and patient outcomes, they also create new legal, privacy, and human rights considerations.
Recognizing these challenges, the Office of the Information and Privacy Commissioner of Ontario (IPC) and the Ontario Human Rights Commission (OHRC) jointly released the Principles for the Responsible Use of Artificial Intelligence (the “IPC-OHRC Principles”). These principles provide guidance for the development, procurement, deployment, and oversight of AI systems used in Ontario's public sector and offer valuable insight for healthcare organizations adopting AI technologies. The IPC has also published AI Scribes: Key Considerations for the Health Sector and an accompanying checklist, which applies the IPC-OHRC Principles to the AI Scribes use case.
In this three-part blog series, we discuss the takeaways from the IPC-OHRC Principles for the use, procurement, and development of AI systems in the healthcare sector. In Part 3, we discuss the guidance provided in the IPC-OHRC Principles for developers of AI systems.
Designing AI for High-Stakes Environments
Healthcare presents a particularly sensitive and heavily regulated environment for AI to operate within. Developers creating products for healthcare organizations must consider technical performance alongside privacy, safety, human rights, transparency, and accountability requirements.
The IPC-OHRC Principles provide a useful framework for developers seeking to create AI systems suitable for healthcare and other public-sector applications. This framework should also be considered by healthcare providers seeking to develop their own AI systems.
Validity and Reliability Must Be Demonstrable
Developers should ensure that AI systems undergo rigorous testing before deployment. The IPC-OHRC Principles distinguish between validity, meaning the system performs its intended function, and reliability, meaning it consistently performs as expected in operational settings. Healthcare clients increasingly expect evidence supporting both claims. Developers should therefore maintain detailed validation records, testing methodologies, and performance data that can be shared with customers evaluating compliance obligations.
Importantly, technical performance alone is not enough. Even highly accurate systems may produce poor outcomes when trained on incomplete, inaccurate, or biased data.
Privacy by Design Should Be Embedded from the Start
Privacy protections are most effective when incorporated during the system design phase. Considering privacy at the design phase can help developers avoid investing in an AI system that may later prove unsuitable for the healthcare context.
The IPC-OHRC Principles encourage developers to embed privacy safeguards through:
Data minimization;
Privacy-enhancing technologies;
De-identification techniques;
Synthetic datasets where appropriate; and
Strong cybersecurity protections.
For healthcare applications, developers should also consider how systems will protect the interests of vulnerable individuals and those unable to provide informed consent.
Addressing Bias and Human Rights Risks
Developers play a critical role in reducing discriminatory outcomes. The IPC-OHRC Principles emphasize the importance of proactively identifying and mitigating bias in training datasets and model design. This requires more than technical testing. It also involves assessing how system outputs may affect different populations and evaluating whether adverse impacts may arise for groups protected under the Ontario Human Rights Code.
Healthcare data often reflects historical inequities in access, treatment, and outcomes. Developers should be prepared to evaluate and address these risks throughout the product lifecycle.
Explainability Is Becoming a Competitive Advantage
For AI vendors, explainability and transparency have become market expectations, not just compliance issues.
Developers should be prepared to provide clear explanations regarding:
How models function;
What data sources were used;
System limitations and risks;
Performance metrics and error rates;
Monitoring procedures; and
Governance controls.
Organizations may find it difficult to satisfy their own legal and ethical obligations if vendors cannot adequately explain system operations.
Supporting Accountability Throughout the Lifecycle
The IPC-OHRC Principles emphasize the need for accountability from design through decommissioning.
Developers should design systems that support:
Ongoing monitoring;
Auditability;
Traceability;
Documentation retention;
Human oversight; and
Safe decommissioning where necessary.
Providing healthcare customers with the tools necessary to monitor and govern AI systems can help strengthen trust and reduce implementation risks.
Conclusion
As healthcare providers continue to adopt AI technologies, developers will be expected to demonstrate not only innovation but also responsible design practices and awareness of the sensitivity of operating in the healthcare sector. The IPC-OHRC Principles make clear that trustworthy AI should be valid, reliable, safe, privacy-protective, human rights-affirming, transparent, and accountable throughout its lifecycle.
Developers that embed these principles into product design will be better positioned to meet client expectations, support regulatory compliance, and contribute to the responsible advancement of AI in healthcare.
Other Parts in this Series:
Part 1: AI in Healthcare: What Providers Need to Know Before Deployment
Part 2: AI Procurement in Healthcare: Key Due Diligence Considerations

