Project Summary

Passion Project
4 People
My Role: Developer & Visual Designer
Designed and developed an AI-enpowered language learning experience that combines tactile and digital interactions.

I led the creative direction, refined the high-fi prototype, developed it with javaScript and Express, created visual branding, and supported the final exhibition.

Final Output

An inclusive Hanzi learning experience developed with AI image recognition.

Why this is not an app?

Project Background

Research shows Chinese learners prefer more tactile and kinaesthetic styles compared to other cultures’ learners. But why?

Hanzi’s originated from oracle bone scripts, which are pictographs and ideographs that represent tangible objects. Chinese Calligraphy is also ingrained in the Hanzi culture not only for functional writing, but also being art from by itself.

Project Goal

How can we integrate the tactile aspects into Hanzi's learning, while also creating an inclusive experience that support different learning styles?  

Brainstorm & Planning

We started by choosing two easy-to-understand Hanzi components. The first one is '木' (meaning 'wood'), where the character's shape graphically represents a tree. The second is '口' (meaning 'mouth'), resembling an open mouth.

The meanings of '口' (mouth) and '木' (wood) extend metaphorically to the human body and nature, respectively. Their combinations give rise to a multitude of unique and meaningful Hanzi characters.

Both elements can serve as base characters that, when repeated, form new characters with related meanings. This tessellation aspect is particularly useful in facilitating graphical learning. Here are some examples:

Traditional Hanzi calligraphy employs a four-grid base ('TianZeGe') to position characters of varying structures.

To make the learning process more intuitive and hands-on, we've decided to utilize this grid system to assists learners to learn about hanzi structures.

Interaction Flow

We've created two paths.

Path 1: When a player assembles an existing Hanzi on the grid, the interface reveals its meaning.

Path 2: If a player forms a new, non-existing Hanzi, they are encouraged to define this new character and can then view similar creations by other users.

We introduced the second path to allow users to contribute to the evolving Hanzi archive. This approach mirrors how Hanzi historically absorbed elements from foreign languages over time. Additionally, this creative exercise fosters a sense of connection among players as they engage in the process.


Tech Component

The tech part is web based. I integrated AI image recognition into the javascript code, and use the machine learning model to read the results of players’ character placement.

The ML model is based on image pixel recognition, so I feed 400+ image samples for each hanzi characters for getting accurate results.

I also set up the backend server to read the users’ data input with Express, and save them to a json file for collecting “New Hanzi Archive”.

Play Testing

We tested our mid-fi prototype with 10 participants.

Insights & Iterations

The beginner participants still feel confused when being prompted to assemble the hanzi building blocks. They also feel the digital experience is a bit disconnected from the physical one. So I made the following iterations:

Final Output

Onboarding Information

1. Introduce the players about what the two base components "木" (wood) and "口" (mouth).

Guided Activity

2. Guide the players to create a hanzi "呆" (dull) to learn about Hanzi structure and how to use the grid "TianZiGe".

Play Time!

3. Players are then invited to create Hanzi by themselves. If they just created an existing hanzi, we show them the meaning of their creations based on Hanzi dictionary.

3.1 If they created something non-existent in Hanzi dictionary, they got to define the new hanzi by themselves!

More projects