SignLens : Bridging the communication gap in real time—hands-free and right in front of the wearer's eyes

About the Project

Over 1.5 billion people worldwide experience hearing loss, including 430 million living with disabling hearing loss. In regions like Malaysia, the crisis is magnified by a critical scarcity of human interpreters—a ratio of just 60 active interpreters for 44,000 Deaf individuals. While conventional camera-based systems struggle with lighting variations and commercial AR glasses cost thousands of dollars, SignLens presents a revolutionary, ultra-low-cost hands-free assistive smart glasses, real-time sign language interpretation. Our system integrates a Raspberry Pi Zero 2 W, a Camera Module V2, and a 0.96-inch OLED optical HUD within a lightweight, custom 3D-printed glasses mount. Operating fully offline to guarantee total user privacy, SignLens executes a four-layer edge pipeline: layer 1 uses MediaPipe for hand tracking to extract 21 key spatial landmark points from each frame creating a 3D skeleton of the hand shape, layer 2 executes a K-Nearest Neighbors (KNN) model for individual letters, layer 3 applies Dynamic Time Warping (DTW) for dynamic signs, and layer 4 construct the whole sentence and display it on the OLED screen. We evaluated across a benchmark dataset of 1,183 samples spanning 24 alphabet classes, SignLens achieved a 99.83% overall classification accuracy, a 1.00 macro average F1-score, and a 120ms median inference latency. By replacing heavy pixel-based neural networks with keypoint vector transformations, SignLens eliminates cloud dependencies, network jitter, and excessive hardware costs. SignLens democratizes assistive technology, removing communication barriers in inclusive classrooms and everyday public interactions while advancing UN Sustainable Development Goals 3, 4, 9, and 10.