Revolutionizing Japanese Calligraphy Learning through AI-Powered Hand Gesture Technology

Revolutionizing Japanese Calligraphy Learning through AI-Powered Hand Gesture Technology
In our rapidly evolving digital era, traditional arts like Japanese calligraphy (Shodo) are often seen as difficult or impractical for the younger generation to learn. To bridge this gap between cultural heritage and modern technology, a new interactive game has been developed: “Hand Gesture Game Kaligrafi Jepang Dengan Konsol Tangan”. Created by Ida Ayu Putu Diah Wahyuni, this innovative project transforms the way people practice Hiragana.
Figure: The research process included direct observation of Hiragana writing patterns to ensure the system’s accuracy.How It Works Unlike traditional methods that require brushes and ink, this web-based application uses Computer Vision and MediaPipe technology to detect hand movements via a standard webcam. Users can “write” in the air using their index finger. The system tracks the finger’s coordinates in real-time and renders them as digital ink on a virtual canvas. To make the experience intuitive, the game uses simple gestures:
• Index Finger Up: Starts the writing mode.
• Grip/Fist Gesture: Stops the writing, simulating lifting a brush from paper.
Smart Evaluation System The game isn’t just about drawing; it’s about learning correctly. Once a user finishes a character, the system performs Similarity Matching. It normalizes the user’s input and compares it against a template of correct Hiragana strokes. Users then receive an instant score and feedback, such as “Perfect,” “Good,” or “Try Again,” making the learning process both fun and effective.By combining traditional art with Human-Computer Interaction (HCI), this project provides a modern, accessible, and paperless way to preserve the beautiful culture of Japanese calligraphy for everyone.