AR | Tokyo Olympics 2020

What's better than an AR filter? An mini game and a face filter!

To match the competitive spirit of the Olympics, we added a game element to this AR effect. This effect features a simon says like memory game sequence. Players are instructed to follow a sequence of poses under a time limit. When the game ends, the player will be rewarded medal(s) and a final score result, showing how many sequences they have completed correctly.

There are a total of 10 pose sequences, the poses are randomized on launch. Initially, we had also incorporated hand gestures, but removed them later because as of v105.1, hand tracking was not supported by IG effects.

Poses were a combination of 3 features: mouth open/close, head tilt left/right, left/right eye blink or both open. Different audio and visual cues are triggered throughout the game.

As always, we push the limits of Spark AR to make the best experience possible. Meta Spark AR Studio is not a game engine, so we had to come up with a custom game state engine system through scripts and patches. With a tight deadline, we had to think outside the box and come up with a solution that would work within the constraints of Spark AR, while still being mobile-friendly.

The final effect was made up of a hybrid script + patch system:

The script is where the game logic and the different game states are setup. This includes the memory game logic, various global variables that have to be initialized and reset from the script and sent to the patches. The script also restarts the game properly and detects if the player has lost or won the game.

The patch editor is where all the visual and audio elements are setup. The mascots' animation controllers are all enabled and reset here, and the script tells the patches which animation the mascot needs to play. Another important job for the patches is to detect if the player has matched the pose. The script tells the pose-matching block which pose the player is currently working on, and determines which thresholds (head tilt angle, mouth open, etc.) the block should be detecting for.

This hybrid system allows for better organization and clearer division between the backend and frontend of the game. Especially since Spark AR isn't a game engine, it's important to utilize both scripts and patches to make life easier when developing.

Tiger Expressions Animation Randomizer

Due to the tight deadline for this project, not a lot of time were spent building an effective debugging system for the gameplay. This meant that play-testing through the entire game took a lot of time as we had to play through the entire game to debug the end state. While some hacks were applied to quicken the gameplay time (screen tap to match pose instead of actually posing), a lot of time were wasted just testing the game. In the future, first week of development should definitely be spent working on an efficient playtesting system.