Your blood test results just came in and three values have an asterisk, so you paste the whole thing into a chatbot like ChatGPT. That isAI in health as patients experience it: useful for understanding, risky as soon as you use it to decide. Here is where the line falls between the two: what is worth asking, what you should never hand over to a screen, and the moment to close it and reach a real person.
On this page
- What the chatbot does well
- Where the conversation goes off track
- What one study shows
- What is best never to paste
- When to close the screen
- How to use it carefully
- Reasonable uses, risky uses
- What clinicians do with it
- Myths and misconceptions
- Frequently asked questions
- Sources
What the chatbot does well
The plain-language explanation is the first thing a patient can reasonably expect from a chatbot. The Collège des médecins du Québec recognizes that AI can give good general health information, and considers it relevant to use it to prepare for an appointment or to understand a diagnosis or treatment already established (Collège des médecins du Québec). The key word is “general”: the chatbot explains well what a term means, much less well what it means for you.
Translating the jargon in a report
A report mentions a “hypoechoic lesion” or “eosinophilia” and the meaning escapes you. Asking the chatbot to rephrase the term gives a first clue, nothing more. What the result means in your case depends on your history and your other tests, and the chatbot has access to none of that.
Preparing your questions before an appointment
This is probably the most worthwhile use: you arrive with a clear list rather than a vague worry. A few useful requests:
- “What questions should I ask my doctor about an abnormal result of this type?”
- “How can I describe my symptoms clearly in two minutes?”
- “What information should I bring to the consultation?”
Going back over an explanation you already received
After a consultation, you sometimes remember only half of it. Having a diagnosis made by a professional rephrased is defensible: the chatbot repeats, it does not decide. The Collège also accepts apps that summarize a visit, provided the note is reviewed.
Where the conversation goes off track
Ahallucination occurs when a tool makes up an answer because it lacks information. It can also leave out something important or misread the data. This applies to consumer tools as much as to those used by professionals (Collège des médecins du Québec).
Confident answers, even when wrong
Smooth, well-structured text gives an impression of accuracy. Yet answers are often delivered with great confidence even when they are wrong, and without expertise in the subject, it is hard to tell good from bad. A confident tone proves nothing.
No examination, no medical record
A chatbot does not see you, does not examine you by touch, does not listen to your breathing. Even with a writing tool, the doctor must add what AI does not capture: the physical exam, nonverbal cues. And most tools do not have access to all of your health information: your allergies, medications and medical history may stay invisible unless you type them in.
Bias and outdated knowledge
According to theWorld Health Organization, these tools can produce false, inaccurate, biased or incomplete statements, and their training data can be biased by race, sex, age or ethnic origin. The organization also describes an “automation bias”: errors that a human would have caught go unnoticed. Canadian privacy authorities also point out that training data can stop at a certain date, so a recent guideline may escape a model.
What a study with real people shows
A randomized study published in the journal Nature Medicine tested what happens when real people, rather than a written test, consult a chatbot (Nature Medicine). It does not try to name the best diagnostic tool: it measures the effect of the conversation, and its authors conclude that none of the models tested was ready for direct patient care.
What the researchers measured
Three doctors wrote ten common medical scenarios. Then 1,298 participants in the United Kingdom were randomly assigned: some consulted one of three major language models, others a source of their choice (control group). Each had to name the possible conditions and choose what to do, from self-care to calling an ambulance.
When the models were given the full scenario themselves, they named at least one relevant condition in 94.9% of cases and the right course of action in 56.3% of cases on average. But participants using those same models named a relevant condition in fewer than 34.5% of cases and the right course of action in fewer than 44.2%. They did no better than the control group on the course of action, and did worse at naming conditions.
Why this gap
The authors point to the conversation itself. In 16 of the 30 exchanges examined in detail, the first message contained only part of the useful information. The chatbot also sometimes suggested the right lead without managing to get it across clearly.
Limits to keep in mind
The models are those available at the time of the study, and the authors say it is unclear whether newer ones would do better with real people. The scenarios were fictional, so without the stress of a real symptom, and the participants lived in the United Kingdom. The researchers recommend testing the tools with real people before offering them to the public. The lesson for a patient: a chatbot that passes an exam does not necessarily guide you well.
To remember
- AI gives good general information, but its answers can be inaccurate or incomplete.
- It can make up an answer, and a confident tone proves nothing.
- It knows neither your medical record nor your body, and it examines no one.
- In the randomized study, participants using a chatbot did no better than the control group.
- No name, no health insurance number and no identifiable image of a result in the chat window.
- Emergency: 911. Non-urgent question: 811. Never stop a treatment on the word of an AI.
What is best never to paste
A piece of personal information is any information that can identify a person, directly or indirectly. The Collège adds that AI is not subject to the same confidentiality rules as a doctor (Collège des médecins du Québec).
Directly or indirectly
Name, address, file number and health insurance number identify you directly. A rare disease or a combination of medical history can identify you indirectly. So removing your name is not always enough: a precise enough medical portrait ends up pointing to someone.
The photo trap
Uploading a photo of a lab report is the most natural and the riskiest thing to do. The Collège cites this example of unintentional sharing: without thinking, you also send the name of the person tested and their address. Retyping only the term or value that puzzles you, without the header, reduces this risk without eliminating it. An isolated value is hard to interpret: the tool sees neither your other results nor your file, and reference values vary from one laboratory to another. Interpretation is up to the professional who requested the test.
Other people’s information
You must absolutely avoid entering information about another person into an AI, because it does not belong to you. The Collège’s example: a specialist’s report that mentions a son’s gambling problems makes that son easy to identify.
Terms of use and the law
Companies may use your words to train their systems, or share or sell them for marketing or profiling purposes. Before choosing a tool, look for four answers in its terms of use: what data is collected, what it is used for, who it is shared with, where it is stored.
On the legal side, Canadian privacy authorities, including the Quebec Commission on Access to Information, published principles for generative AI, then a joint resolution on the responsible use of AI in health care. These principles place responsibility on the organizations that develop, offer or use these tools, and they include limiting the disclosure of personal, sensitive or confidential information. The Act respecting the protection of personal information in the private sector, modernized by Law 25, governs these companies: after a joint investigation of ChatGPT, the Commission concluded that the company had not fully met its obligations, and the company committed to further measures. This law does not replace your own caution: according to the Collège, an informed person remains free to enter their own information, provided they weigh the risks. See also our article on protecting digital health data.
When to close the screen
A Urgency cannot be settled by texting a machine. Some situations call for a human right away, others at least for a real conversation before deciding.
911: without going through the chatbot
Sudden trouble breathing, chest pain, tightness or pressure, signs of a stroke (drooping face, weakness in one arm, trouble speaking), loss of consciousness, a severe allergic reaction, bleeding that will not stop, a suspected overdose, a life in danger: call 911 instead of writing a description for a tool. If in doubt, call.
811: the non-urgent question
For a concern that is not an emergency, Info-Santé 811 puts you in touch with a nurse. See what the Info-Santé 811 line does. On the line, a person asks questions and guides you. If you are thinking about suicide or are in deep distress, do not stay alone with a screen: Info-Social answers at 811, option 2, and the 1 866 APPELLE (277-3553) line is open day and night. If a life is in danger, call 911.
A consultation, when the doubt remains
A symptom that comes back, a result that worries you, a chatbot answer that contradicts your professional: these are reasons to consult. If you are unsure which door to open, see how to choose between the emergency room and a consultation.
The Collège is clear: even if an AI advises it, never stop a treatment without talking to a doctor or another health professional.
A question the screen cannot settle? Make an appointment or consult a health professional online, throughout Quebec.
How to use it carefully
A careful use relies on a few habits, before, during and after the conversation.
Before you type
- Remove the name, address, health insurance number, file number and anything in a document’s header.
- Talk only about yourself, never about another identifiable person.
- If you have a choice, prefer a tool offered by a recognized health organization (the Ministère de la Santé et des Services sociaux or Santé Québec) or recommended by your doctor, as the Collège suggests.
During the conversation
Give only the context your question needs (how long, what other conditions, what medications), without piling up details that could paint an identifiable picture of you. Ask the chatbot what is missing for its answer to be reliable, and treat each statement as a lead, not a verdict.
After the conversation
If the answer touches on an important health matter, the Collège considers it essential to talk to a doctor or another professional to check the information. Write it down and bring it to the appointment. To cross-check a specific fact, our article on how to sort through false health information offers simple guidance.
Reasonable uses, risky uses
The table below sums up the dividing line, in the spirit of the Collège’s recommendations. If in doubt, the last column prevails.
| Use and level of risk | What to do instead |
|---|---|
| Having a word from a report explained: reasonable, with no name or number | Have the person treating you confirm the meaning |
| Preparing a list of questions before an appointment: reasonable | Bring the list and read it during the consultation |
| Guessing the cause of a new symptom: at risk : no examination, no medical record | Call 811, or consult if the doubt remains |
| Uploading a photo of a lab result: at risk : your name and address go with it | Ask your health professional to explain it; if you still use the tool, retype only the term or value, without the header |
| Deciding to stop or change a medication: at risk : never without advice | Talk to a doctor or another health professional |
| Summarizing your visit with an app: possible, with review | Correct the note and tell the doctor you used it |
| Describing a serious or sudden situation: at risk : every minute counts | Call 911 |
What clinicians do with it
The health professionals also use AI, within a framework the patient does not control. The Collège cites writing notes, assessing risks, enhancing radiology images and scheduling visits (Collège des médecins du Québec). For an overview of the health system, see our articles on artificial intelligence in the health system and on new technologies and the health system.
Responsibility remains human
Doctors remain fully responsible for signed diagnoses, treatments, notes and reports, with or without AI, and clinical judgment must prevail (Collège des médecins du Québec). The Collège adds that a doctor who cannot verify information produced by an AI should not use it, and that entering identifying information into an uncertified consumer tool exposes that data to the risk of leaks (Collège des médecins du Québec).
Your rights when AI enters the consultation
When an AI captures your voice or image, the doctor must obtain your consent. You can refuse, and the doctor then writes the note without the tool. If in doubt, ask whether the tool is certified by the Bureau de certification de Santé Québec.
Tools that claim to be medical
In Canada, medical devices based on machine learning fall under the Food and Drugs Act and its Medical Devices Regulations, and their manufacturers must demonstrate their safety and effectiveness according to the guidance from Health Canada. If an app claims to make a diagnosis, check whether it appears on Health Canada’s list of licensed medical devices, and have its answer verified by a doctor or another health professional.
Myths and misconceptions
“A chatbot that passes medical exams can guide me as well as a professional”
False. In the Nature Medicine study, the models alone named at least one relevant condition in 94.9% of cases, but participants who consulted them did so in fewer than 34.5% of cases and chose the right course of action in fewer than 44.2%.
“If the answer is well written and confident, it is probably right”
False. The Collège writes that AI answers are often given with great confidence even when they are wrong.
“What I write stays between the chatbot and me”
False. Confidentiality is not guaranteed with consumer tools, and AI is not subject to the same rules as a doctor.
“If I remove my name, my data is anonymous”
Nuanced. Removing your name and health insurance number reduces the risk, but a rare disease or a combination of medical history can identify a person indirectly.
“A doctor who uses AI lets the machine decide”
False. Doctors remain fully responsible for their actions, with or without AI, and the tool serves only as a decision aid.
Frequently asked questions
Can you describe your symptoms to a chatbot?
The Collège des médecins du Québec recommends avoiding AI for self-diagnosis, except for a very simple problem with no major consequences. For a worrying symptom, Info-Santé 811 or a consultation is safer, and 911 in an emergency.
What information should you never paste into a chatbot?
Anything that identifies a person, directly (name, address, file number, health insurance number) or indirectly (rare disease, combination of medical history), as well as anything about another person and photos of reports with a header.
Can I have a chatbot explain my blood test results?
You can ask what a term means, once everything that identifies you has been removed. The tool does not know your medical record: interpreting your results is up to the professional who ordered them.
What should I do if the chatbot reassures me but I am still worried?
Trust your concern rather than a written answer: call 811 for a non-urgent question, consult a health professional, or call 911 if the symptom is serious or sudden.
Can my doctor use AI during my consultation?
Yes, under certain conditions. The doctor remains responsible for decisions and notes, and must obtain your consent when an AI captures your voice or image. You can refuse, and the doctor then writes the note without the tool.
Sources
- Collège des médecins du Québec, Health care and artificial intelligence (information for the public, in French)
- Collège des médecins du Québec, Artificial intelligence in medicine (in French)
- Collège des médecins du Québec, Accountability and professional responsibility of doctors (AI systems, in French)
- World Health Organization, news release on ethics and governance guidance for large multi-modal AI models
- Commission d’accès à l’information du Québec, principles for the development and use of generative AI
- Health Canada, pre-market guidance for machine learning-enabled medical devices
- Nature Medicine, Reliability of LLMs as medical assistants for the general public: a randomized preregistered study
- Santé Québec, When to go to the emergency room (in French)
- Heart & Stroke, Emergency signs: when to call 9-1-1 (in French)
- Commission d’accès à l’information du Québec, results of the joint investigation of ChatGPT (in French)
- Commission d’accès à l’information du Québec, joint resolution on the responsible use of AI in health care (in French)
- Office of the Privacy Commissioner of Canada, principles for responsible generative AI technologies (in French)
- Private Sector Privacy Protection Act
- Health Canada, Medical Devices Active Licence Listing (in French)
Omicron Clinic
Need to consult a doctor?
In-person or via telemedicine, anywhere in Quebec.
We accept insurance. Telemedicine consultations available 7 days a week. No family doctor required.
