Built on a
policy framework
In My Words is a pilot built for a course on explainable AI, and it's an attempt to put a policy framework for responsible AI in mental health care into practice.
The gap
Therapy sessions with young people often start cold, with forms or direct questions, and most young people don't describe their lives in clinical terms. Some play down how they're feeling, and some haven't found the words for it yet.
In My Words lets a young person write first, in their own words, and then see exactly how it will be presented before anyone else reads it. They do it before every session, so the clinician walks in knowing what's on their mind, and each conversation can start from there.
What it isn't
- It isn't a chatbot, and the AI never replies to the young person.
- It doesn't diagnose, score, or predict risk.
- It isn't a crisis service, and nobody watches it in real time.
- It isn't ready for real patients, since it hasn't been tested or approved for clinical care.
Promoting Equitable Access to Mental Health Care Through Responsible AI Implementation
The policy brief makes five risk-based recommendations, and each one shaped part of the tool.
- 1
Prioritize appropriate, low-risk uses
The AI does one narrow job, organizing what a young person wrote before a session with a person. It doesn't chat, give advice, diagnose, or estimate risk.
- 2
Adapt locally, with community input
The questions are modeled on check-ins young people fill out before each therapy session, and young people's slang and dialect are quoted exactly as written. The next step is testing the wording with young people themselves.
- 3
Keep clinicians accountable
The young person approves the brief before it's sent, and the clinician approves the plain-language plan before the young person sees it.
- 4
Protect and govern data
Only what the young person sends is saved, and anything they remove is deleted. The one thing kept across check-ins is their 0 to 10 rating and reason, which they and their clinician can see over time. Before they begin, they're told it follows the same privacy rules as the clinic, and where confidentiality ends.
- 5
Evaluate before scaling
It has been tested on 20 fictional young people built from real, cited quotes, and its known limits are listed openly below.
Data and testing
No real patient data was used. The test cases are 20 fictional young people, and every line they write is adapted from a real quote by an adolescent in published research, mostly open-access qualitative studies. Sources with restrictive licenses were left out.
The personas were chosen to cover the different ways young people talk about themselves. Some are guarded and some share a lot, some have been in therapy before, some were never taught to talk about feelings, and several are LGBTQ+ young people describing themselves in their own terms.
Known limits
- The safety rule looks for specific phrases, so indirect wording like "eventually we're all gonna die" can get past it. The AI is asked to flag unclear phrases, and clinicians should still screen the way they normally would.
- Very little research has tested written pre-visit prompts for young people, so most of the wording is adapted from spoken interviews, surveys, and design codes.
- The examples of young people who say very little come from studies in Norway, the UK, and Germany, because I couldn't find verified US examples.
- Voice typing in this demo uses the browser's speech service, so the audio goes to Google in Chrome or Microsoft in Edge. That's for demo speed only. A real deployment should transcribe on the device, for example with an open model like Whisper running in the browser.
- Pictures are checked by AI before they're stored. A real deployment would also need industry-standard hash matching for child sexual abuse material, the reporting process U.S. law requires of services that host images, and a legal review.
- It hasn't been tried with young people or clinicians yet. The planned next step is a think-aloud test with 18- and 19-year-olds.
Built for Explainable AI, Duke University, Fall 2026.