SenseBridge

Flutter app for disability assistance. It runs Auslan sign translation, obstacle detection, and noise detection on-device with TFLite models. Built for Unihack 2026; received the Honorable Award for Social Impact.

What it does

• Translates Auslan hand signs to text from the camera feed
• Detects obstacles and vehicles and gives visual and audio alerts
• Listens for environmental sounds and warns with vibration and on-screen messages
• Runs inference on the phone so it does not need a constant cloud connection

How it works

Flutter/Dart client with three TFLite modules:

1. Auslan translation

• Camera → Auslan TFLite model → recognized sign → text on screen

2. Obstacle detection

• Camera → YOLOv8 / SSD MobileNet TFLite models → obstacle and vehicle boxes → visual/audio alerts
• Models fine-tuned on custom road-scene data to cut false detections

3. Noise detection

• Microphone → sound classification model → vibration and on-screen warning

lib/
  main.dart
  app/
    sense_bridge_home.dart   # Main navigation
    screens/                 # Feature UIs
    feature/                 # ML integration
    widgets/
assets/
  auslan_detection/          # Auslan TFLite model
  obstacle/                  # Obstacle models & labels
  noise_detection/           # Noise model & CSVs
backend/                     # Optional Python model serving

Results

Competition

• Unihack 2026: Honorable Award for Social Impact

Measured outcomes

• YOLOv8 and SSD MobileNet tuning improved obstacle/vehicle detection accuracy by about 20% on mobile
• Fine-tuning on custom road-scene data reduced false detections
• Flutter pipeline ran inference under 100ms latency

Tech stack

• Mobile: Flutter / Dart
• ML: TensorFlow Lite, YOLOv8, SSD MobileNet, sound classification
• Sensors: camera, microphone, vibration
• Backend (optional): Python model serving
• Deploy: Render (render.yaml)

Project link: github.com/narolinelim/sense_bridge