Accessibility
Subtitles that let HOH viewers immerse
in the movie’s emotions & context
HOH viewers’ satisfaction improved by 153%
Role
Primary Research
Secondary Research
Co-design Workshop
Wireframe
User Interface
Usability Test
Solo Project
Personal Project
(2025.07~2025.09)
Impact
Satisfaction increased by 153%
Why I Care About Accessibility
Struggling to Grasp Movie Context During Temporary Hearing Loss
I once experienced sudden hearing loss and felt the discomfort of missing everyday sounds. Since I love movies, I tried watching them with subtitles, but I couldn’t fully immerse myself. This naturally led me to think about the movie-watching experience of deaf viewers and inspired this project.
Problem
Current subtitles failed to convey enough emotion and context.
HOH viewers find it hard to immerse because subtitles don’t fully convey emotions, context, and sound storytelling, making it difficult to enjoy movies on the same level as hearing viewers.
Solution
Filling in missing sounds with visual captions, haptics, and vibration
With subtitle presets and haptic feedback, the experience went beyond basic understanding to supplement emotion and context, enabling them to enjoy movies on the same level as hearing viewers.
Current Streaming Platforms
Streaming platform subtitles often fail to convey background music or the actors’ emotions.
I’m okay
I didn’t realize the male lead was so nervous when he confessed.
Emotions from the actor’s voice
are not conveyed
( The victim felt fear )
Actually, they said a gunshot was heard in the background.
If the camera isn’t on the sound source, the sound is hard to understand
Interview Goal
How do gaps in subtitles affect hard of hearing viewers
fully immerse in and enjoy the emotional journey of a movie?
Interview Outline
Topic
How do gaps in subtitles affect immersion and emotional experience for deaf and hard-of-hearing viewers?
Method
Mixed: Google Meet (online) + in-person
Participants: 4 HOH viewers
Questions on whether
Standard subtitles miss important elements
Gaps affect immersion and emotional understanding
Improved subtitles change the viewing experience
The amount of information feels overwhelming
Interview Insight
( John is running )
Unwanted subtitle spoilers reduce enjoyment.
What, I got spoiled!!
Limited sound immersion weakens emotional impact.
So that was the sound of cards? I didn’t know.
Madness is the only freedom left.
Need to ask someone when not understood in the middle.
What did he just say? 😥
Want the same level of understanding as non-disabled users.
“I want to be surprised, cry, and laugh at the same moments and directions.”
Current subtitles make it hard to enjoy the movie while following the content.
“I need to be in the right mindset to watch a movie.”
Literature Review
Visual, customizable subtitles and haptic make movies clearer for HOH users.
Font
Visual Subtitles
For music, visual expressions such as color, font, and position are more effective than plain text.
Ahn, S., Kim, J., Shin, C., & Hong, J.-H. (2024). Visualizing Speech Styles in Captions for Deaf and Hard-of-Hearing Viewers. International Journal of Human-Computer Studies, 103386–103386. https://doi.org/10.1016/j.ijhcs.2024.103386
Customizable Subtitles
Deaf & hard-of-hearing users prefer customization, and current subtitles lack representations of sound, emotion & distinction.
Uzzo, G. (2025). “Your subtitles will look like this”: Exploring user preferences for closed captions across streaming platforms. The International Journal of Translation and Interpreting Research, 17(2), 55–72. https://doi.org/10.12807/ti.117202.2025.a04
Haptic Feedback
Deaf & hard-of-hearing users can perceive sound information through vibration, so vibration assistive devices are useful.
Mirzaei, M., Kán, P., & Kaufmann, H. (2021). Effects of Using Vibrotactile Feedback on Sound Localization by Deaf and Hard-of-Hearing People in Virtual Environments. Electronics, 10(22), 2794. https://doi.org/10.3390/electronics10222794
Initial Ideation
Ideated research-based methods for intuitive understanding of movie sounds.
Question
From the HOH perspective, I wondered
if too much information might disrupt immersion.
Codesign Workshop
Goal
Finding balance that helps HOH viewers
understand emotions & context without breaking immersion.
Immersion
Rich understanding
Participants: 6
Duration: 90 min each
Format
Codesign Workshop
Process
Icebreaking
Identify easily missed sounds
Empathy mapping of discomforts
Assess the level of concreteness in expressions
Test expression methods
Research preferences for descriptive subtitles
Footsteps
Laughter
Crowds
Shift in musical tone
Glass breaking
Door Closed
Breathing
Phone vibration
Speech rate
Rain
No subtitles → mismatch in lip-reading and eye focus
Spoiler exposure (OTT upload delays, hard to join conversations)
Lack of data on sound effects experience
Fatigue (hard to tell who is speaking / identify sounds)
Insufficient subtitle descriptions
Subtitles moving too fast
Spoiler subtitles
Emotional direction not conveyed
Asking questions pattern in real cinemas
Desire to have the same level of understanding as non-disabled viewer
Inconvenience of frequent rewinding
If you experienced any of the following, please rate your level of discomfort:
1 = Not very uncomfortable
3 = Uncomfortable
[Additional comments]
Simple: [Door closes], [Footsteps] / Line: [You really wanna get scolded?]
Medium: [Door slams, room echoes] / Line: [You really wanna get scolded?]
Rich: [Door slams, room echoes] + Haptic feedback + Sound infographic /
Line: [You really wanna get scolded?] with intonation marks + timed text (“Step step”)
Glasses temple (earpiece)
Electronic earrings
Apple Watch / Galaxy Watch
Electronic ring (e.g., Galaxy Ring)
Wireless earphones (AirPods)
Other ( )
Explanation style
Simple Interest Curve
Medium Interest Curve
Rich Interest Curve
Time
Time
Time
Degree of interest
Degree of interest
Degree of interest
Which of the three subtitle styles do you think gives the best immersion and enjoyment?
If vibration feedback could be received while watching
a movie, which device would you prefer for haptic feedback?
Drawing additional details
What do you think about turning movie subtitles into variety-show style captions?
Level 1 (Monochrome)
Level 2 (Text styling)
Level 3 (Fancy)
Preferred / Not preferred
Preferred / Not preferred
Preferred / Not preferred
[Door slams, room echoes]
You really wanna get scolded?
Boss! I’ll really do betTer!!
I didn’t catch that just now.
I didn’t catch that just now.
step
step
Bang!
Step
step
Did you get what Shina said?
What?
Keep (Good):
Adapt (Good but needs changes):
Stop (Not good):
I didn’t catch that just now.
AI
What did you just say?
*Permission to use interviewee photos in this portfolio has been obtained.
Current subtitles score : 1.9
Q. How would you rate your satisfaction with watching movies on OTT platforms?
Strongly dissatisfied
1
2
Neutral
3
Satisfied
Dissatisfied
4
5
Very Satisfied
Don’t
Do
Distracting!
Subtitles are positioned where the speaker is located
Subtitles with too much detail
Hard to tell movie from graphics!
Graphics that ruin the movie
Asking in the middle ruins immersion
Interactive AI search while watching
AirPods conflict with hearing aids, not usable
The vibration feels too simple and repetitive
Keep subtitles at the bottom,
avoid variation
Describe only the current scene
Use only vibration as an extra
immersive cue
Review unclear parts after watching
Watch works best for haptic feedback
Detailed vibration feedback for each situation
User Flow
STEP 1
Open a movie on Netflix
STEP 2
Launch our app
STEP 3
Search for the movie title
Click!
Search
Click!
Final Design
Subtitles express emotions and context,
covering non-verbal and paralinguistic cues
Conveying sound vibrations
to the users via a smartwatch
Customizable subtitle settings adjust to user preference
Genre-based templates
enable quick subtitle choice
Musical
( 🎹 Piano melody, enjoyably playing)
I will be back!
Action
Run!!!
(Boom💣!)
Drama
No..Don’t come here..
You are dead.
Romance
What are you drawing?
Bookmark scenes to
rewatch when sound is unclear
User Feedback
*Permission to use interviewee photos in this portfolio has been obtained.
Compared to existing OTT subtitles, the current version is better.
Strongly Disagree
1
2
Neutral
3
Agree
Disagree
4
5
Strongly Agree
The app makes it easier to understand a film’s emotions and context.
Strongly Disagree
1
2
Neutral
3
Agree
Disagree
4
5
Strongly Agree
The app does not disturb immersion when watching a movie.
Strongly Disagree
2
Neutral
3
Agree
Disagree
4
5
Strongly Agree
“Naturally got immersed as it conveyed emotions and sound effects missing in regular OTT subtitles.”
“Since it was customized, extra information appeared only when needed and didn’t break the flow.”
Impact
Satisfaction increased by 153%
Q. How would you rate your satisfaction
with watching movies on OTT platforms?
Satisfaction score with
Regular OTT subtitles
1.9
8 respondents
2.5X
Q. What is your satisfaction level with
the improved subtitles versus the original?
Satisfaction score with
Talebridge subtitles
4.8
8 respondents
UI System
Built the design system from start to finish
Accessibility
Integrated a11y checks to ensure accessible color contrast and readability
What I’ve learned
Enhancing Immersion Without Distracting
I realized that designing for accessibility means delivering the necessary information with clarity while preserving immersion without disrupting the flow of the movie. In particular, through the process of designing subtitles for deaf viewers, I learned that the effectiveness greatly depends on how visual and haptic elements are arranged to supplement emotions and context. Ultimately, accessibility design is about carefully balancing clarity and immersion, and I came to understand that achieving this balance is the true starting point for improving user experience.
More projects
Business Strategy
Built Personality-Level Trust in Dating Experiences
Achieved 40% increase in users
Sponsored project, Hardware
Ideation Meeting Support Tool for ADHD
Helping Users 59.3% More Focused
Hardware Interaction
Baby cry cause analysis using a wearable device and app
Helping understand babies better
