From First Hackathon to CSIRO: Building AI for Sign Language Learning

Published by

on

The Idea

Summer 2024. I joined my first-ever hackathon at Victoria University, focused on accessibility. One question drove me: What if AI could make sign language learning more accessible?

Research revealed the problem: Australia has a critical shortage of Auslan interpreters and educators, creating real barriers for the deaf community.

Building the Prototype

The challenge: I needed a dataset, but none existed for my use case.

The solution: I built a quick local app to record videos and asked family and friends to help. Goal: 20+ videos per gesture across 4 basic gestures.

The process: I trained a model using Python libraries on my MacBook. It worked , the model detected hand movements based on my collected data.

What I Learned

Training your own model is tough:

  • Requires massive amounts of data for accuracy
  • Data collection takes forever
  • Model training needs serious GPU power (my MacBook’s CPU maxed out after one training run)

The big realization: Knowing what I know now, hyperscalers like OpenAI and Google offer models that are good enough for most use cases. You only need custom training when no dataset exists or you have very specific needs.

The Journey

Won the Victoria University hackathon

Selected for CSIRO’s ON Prime pre-accelerator cohort with help from my course chairperson

Learned market validation by interviewing real users and testing assumptions

Full circle moment: Last month, one of my market validation interviewees became a guest speaker at our company event

The Takeaway

It’s crazy how far one idea can go!!


Check out EduSign: GitHub Repository


Discover more from Ivy Fiecas-Borjal

Subscribe to get the latest posts sent to your email.

Discover more from Ivy Fiecas-Borjal

Subscribe now to keep reading and get access to the full archive.

Continue reading