Smart Tour Guide — AR Information Assistant

Smart Tour Guide (STG) combines computer vision and voice technologies with a wearable AR experience to deliver on-the-spot knowledge about places and artifacts. The system recognizes landmarks, translates ancient texts, identifies historical figures, and provides an interactive voice assistant — enabling a guided, informative tour without a human guide.

• P1 — STG glasses: embedded prototype and hardware integration
• P2 — Pharaonic languages translation: hieroglyphic text understanding/translation
• P3 — Pharaohs face recognition: identity recognition for statues and portraits
• P4 — Landmarks recognition: scene and POI detection/localization
• P5 — Voice assistant: Q&A, weather, and multilingual translation
• P6 — Sign language translation: hand-sign classification for accessibility

A hardware-centric prototype of smart glasses built on microcontrollers, providing core on-device capabilities: camera capture, lightweight CV, basic UI feedback, and wireless connectivity.

• Capabilities:
  • Camera capture and local inference on MaixDuino/MicroPython class devices
  • On-device prompts/alerts; simple display or audio output
  • Network connectivity for cloud-assisted tasks
• Tech:
  • MaixDuino, MicroPython, embedded CV, basic networking
• Why it matters:
  • Establishes the wearable form factor and real-time constraints for user experience

Translates ancient Egyptian (hieroglyphic) text to modern language. Pipeline includes script region detection, symbol segmentation, sequence modeling, and post-processing.

• Capabilities:
  • Detect and segment hieroglyphic glyphs from photos
  • Sequence modeling to map glyphs to phonetic/semantic units
  • Language translation to readable modern text
• Tech:
  • YOLO for detection; custom arrangement/modification algorithms for ordering
  • OCR/sequence approaches for symbol-to-text, translation layers
• Why it matters:
  • Unlocks context of historical inscriptions directly on-site

Identifies kings/queens and notable figures from statue images or portraits to retrieve biographical summaries.

• Capabilities:
  • Face detection and alignment on statues/reliefs under challenging conditions
  • Face embeddings and classification to known identities
  • Info overlay: name, era, achievements
• Tech:
  • OpenCV, face_recognition, YOLO-based detection
• Why it matters:
  • Turns visual artifacts into interactive, educational experiences

Detects and localizes landmarks to provide on-screen overlays with names and facts, allowing context-aware guidance.

• Capabilities:
  • Landmark detection in real-world scenes
  • Non-maximum suppression for clean overlays
  • Overlay of titles, directions, and brief descriptions
• Tech:
  • YOLO detectors, NMS, classical CV utilities
• Why it matters:
  • Helps visitors orient themselves and discover key points quickly

A conversational assistant that answers tourist questions, reports weather, and translates between languages — hands-free and on the go.

• Capabilities:
  • Speech-to-text (STT) and text-to-speech (TTS)
  • Open-domain Q&A for museum/landmark facts
  • Weather and utility skills; language translation
• Tech:
  • gTTS, speech_recognition, playsound, openweather API, googletrans, basic NLP
• Why it matters:
  • Delivers a natural interface that reduces friction and boosts accessibility

Recognizes hand signs that map to characters/words, supporting basic communication for deaf or hard-of-hearing users.

• Capabilities:
  • Real-time hand landmark extraction and sign classification
  • Spells characters/words; can feed the voice assistant
• Tech:
  • MediaPipe Hands, classical ML classifiers
• Why it matters:
  • Expands inclusivity for museum and outdoor tour experiences

• On-device workflows prioritize low latency; heavier models can offload to edge/cloud
• AR overlays complement audio guidance; both adapt to ambient light/noise
• Modular design enables deploying subsets (e.g., only P3 + P4) per venue

• Source: https://github.com/Abdelraouf-Hawash/Smart_Tour_Guide