Newsletter
This A.I. Project Lets Your Hands Do the Talking
Deploying richer sign language 'data'
All the world’s abuzz about A.I., which can translate whole treatises into different languages and could even make dubbing irrelevant.
We’ve come far in terms of closing the communications gap between languages. Even before ChatGPT hit the scene, Google Translate had been able to help facilitate conversations between different language speakers, or translate signage in real-time.
But what about the hearing-impaired? Tech’s been a bit slower to facilitate communications on their behalf.
Enter Fingerspelling.xyz and GiveAHand.ai.
For the American Society for Deaf Children (ASDC), creative studio HelloMonday/DEPT created a means to build sign language databases … and now, deploy them to accelerate comprehension IRL.
The sites work in tandem. Fingerspelling.xyz, which Hello Monday/DEPT and ASDC launched in 2021, uses machine learning to help others comprehend the sign language alphabet. The platform is now part of the ASDC’s training materials.
The novelty here is GiveAHand.ai. Entirely crowdsourced and launched on American Sign Language Day (April 15), it’s the world’s biggest dataset of hand signs, including diverse shapes, backgrounds, colors and gestures.
Sign language is a tough subject for live audio transcription and translation tools to grasp and transcribe. There are different dialects, plus rapid hand gestures, facial expressions and full body movements. Such elements weave a web of complexity and, sometimes, add layers of complication.
People can contribute more of this nuanced physical “data” to GiveAHand.ai, building a rich library that allows researchers and companies to build better models for democratizing communications accessibility across a broader swathe of the populace.
We’ve heard it said that 90 percent of communications is non-verbal. So much depends on looks, motion and posture, after all. While that anecdote doesn’t strictly apply to sign language, there’s something re-humanizing about plugging the nuances of gesture into the algorithmic universe—excising the ghost, and adding more soul to the machine.
Angela Natividad is the European markets editor at Muse by Clio. She also writes about gaming and fashion, and whatever else she’s interested in, really. She’s based in Paris and North Italy, so if you’re local, say hi. She might eat all your food.
