Can you use AI scribes in healthcare while protecting PHI?

Are AI scribes legal under Canadian health privacy law?

AI scribes are legal in Canadian healthcare, but they are not automatically compliant. Recording a patient encounter, transcribing it, and generating a clinical note are all collections, uses, and disclosures of personal health information, so every rule that governs PHI handling applies to the AI tool and its vendor.

Provincial health privacy laws set the baseline. In Ontario, PHIPA requires a health information custodian to take reasonable steps to protect PHI and remain accountable when an agent or service provider processes that information. Quebec's Law 25, British Columbia's PIPA and FOIPPA, Alberta's HIA and PIPA, and federal PIPEDA each impose comparable accountability, consent, safeguarding, and transparency duties. The custodian, not the software company, is ultimately answerable to the regulator.

The practical takeaway is that adopting an AI scribe does not transfer your legal obligations to the vendor. You are deploying a sub-processor of highly sensitive data, and you must be able to show a regulator that you assessed the risk, controlled it, and documented your decision.

What safeguards make an AI scribe PHI-safe?

An AI scribe is PHI-safe only when consent, contracts, technical controls, and human oversight all line up before the first patient is recorded. No single control is sufficient on its own.

• Valid patient consent: inform patients that an AI tool is recording and drafting the note, explain how the recording is used and retained, and offer a way to decline without affecting their care.
• A written agreement with the vendor: define them as a service provider or agent, with binding obligations on confidentiality, breach notification, audit rights, subcontractor disclosure, and return or deletion of data on termination.
• No training on your data: confirm in writing that the vendor will not use your audio, transcripts, or notes to train or improve its models, and that prompts and outputs are not retained for vendor purposes.
• Data residency and flow mapping: know where audio and transcripts are stored and processed, confirm Canadian residency where your contracts, sector, or provincial expectations require it, and document any cross-border transfer.
• Encryption and access control: PHI must be encrypted in transit and at rest, with role-based access, strong authentication, and audit logging on both your side and the vendor's.
• Data minimization and retention limits: capture only what the clinical note needs, delete raw audio promptly once the note is finalized, and set defined retention periods rather than indefinite storage.
• Human review in the loop: a clinician must read, correct, and approve every AI-generated note before it enters the record, because generative models can omit, invent, or misattribute clinical detail.

Why do you need an AI Privacy Impact Assessment for a scribe?

You need an AI Privacy Impact Assessment (AI-PIA) because an AI scribe combines two of the highest-risk factors in privacy: sensitive health information and an automated system whose behaviour is hard to fully predict. An AI-PIA is the structured process that surfaces those risks before deployment and gives you a documented, defensible record of your decision.

A general PIA examines how a program affects individuals' privacy against established principles such as accountability, limiting collection, safeguards, retention, and individual access. An AI-PIA layers on the questions unique to machine-learning systems: what data trains or fine-tunes the model, whether outputs can be inaccurate or biased, how decisions are explained, whether the vendor reuses your data, and what happens when the model produces a wrong or fabricated note.

It is worth being precise about who is legally required to do a PIA. Federal and provincial public bodies face statutory mandates; for example, BC's FOIPPA requires public bodies to complete a PIA during development and before a new system launches, and the federal Treasury Board Secretariat (TBS) Directive on PIA requires assessments for programs that use personal information in decisions affecting individuals. Most private clinics and health-tech vendors are not bound by those specific government mandates, but the methodology is best practice and is increasingly demanded by hospitals, health authorities, and enterprise buyers as a condition of doing business. The federal Office of the Privacy Commissioner describes a PIA as a risk management process and states plainly that it does not approve, endorse, or sign off on PIA reports, so the responsibility to get it right stays with you.

How do you vet an AI scribe vendor before you deploy?

You vet an AI scribe vendor the way you would vet any processor of sensitive data: with documented due diligence that tests their security posture, data practices, and contractual commitments before any PHI flows. A polished demo is not evidence of compliance.

• Request independent assurance such as a SOC 2 Type II report or ISO 27001 certification, and read the scope and exceptions rather than accepting the logo at face value.
• Ask exactly where data is stored and processed, which subcontractors and foundation-model providers are involved, and whether any of them can access your PHI.
• Get the no-training and no-data-reuse commitments in the contract, not just in marketing copy or a sales email.
• Confirm breach notification timelines, deletion and data-return procedures, and your right to audit or receive evidence of controls.
• Test accuracy and failure modes: how often the scribe mis-hears, hallucinates, or drops clinical detail, and what guardrails exist when it does.
• Check that retention, access logging, and patient-decline handling can be configured to match your own policies, not the vendor's defaults.

What are the biggest risks if you skip the privacy work?

The biggest risks of deploying an AI scribe without privacy work are inaccurate clinical records, unconsented surveillance of patients, uncontrolled cross-border data flows, and a vendor quietly training its models on your patients' conversations. Any one of these can trigger a reportable privacy breach, regulator scrutiny, loss of patient trust, and clinical harm if a flawed note drives a care decision.

Two failure patterns are especially common. First, organizations assume the vendor's security covers their compliance, when accountability for PHI legally remains with the custodian. Second, teams adopt the tool informally, clinician by clinician, with no consent process, no contract review, and no record of the decision, which is exactly the gap a regulator or hospital procurement reviewer finds first.

The good news is that these risks are manageable. With consent, a vendor agreement, an AI-PIA, the right technical controls, and human review of every note, an AI scribe can reduce clinician burnout and documentation burden while keeping PHI protected and your organization defensible. Privacy Horizon helps healthcare and health-tech teams run that assessment and stand up the controls, so you can adopt AI tools with confidence rather than crossing your fingers.