Back to Work

AR | Meta x Tiffany & Co.

2023

Tiffany & Co. Hero Image

The Landmark

In 2023, Tiffany & Co., one of the most luxurious jewelry design house, came to us to help them design and develop an AR experience to celebrate Tiffany's flagship relaunch on Fifth Avenue in Midtown, Manhattan.

As the lead creative technologist, I prototyped initial demos and technical demonstrations for our internal creative team and continued to develop the experience until the end.

The experience highlights some of the most intricate animated 3D jewels, which was a challenge to reflect in AR while keeping the experience under Meta’s AR effect file size restrictions. Because of this, we

This experience has many movement parts real-time animations throughout, post-processing effects to add dynamic sparkling to the gems, and a fully immersive environment takeover.

The User Flow

User starts off on the back camera, which then triggers the animations to begin. A couple of bejeweled fish, along with a few other animals, animate in and lights up each floor of the main Tiffany landmark building. The roof of the landmark then lifts to reveal a giant spinning Tiffany diamond. The user is then encouraged to swipe across their screen to rotate the diamond some more. A series of animations will progress as the user looks around the scene and immerses themselves in this fantasy Tiffany intersection. At this point, the user can also switch to front camera for a selfie mode, where everything looks the same except placed farther back behind the user now. All animations in the scene will continue looping throughout the effect.

Since the effect simulates a 3 DoF experience, the user can rotate their camera to view more of the scene. But the camera feed will start to fade in once user rotates past the thresholds set.

Tiffany & Co. User Flow

Technical Highlights

Quick note: At the beginning, client was able to grant us an additional 3MB of file size to work with, making the file size limit 7MB instead of the usual 4MB! While this seemed super generous, it soon became apparent to us that it was still not enough to fit all the assets in the effect to match our original style ref. So we were able to push for a 10MB file size limit!

Tiffany Diamond's Kira Effect

Aside from the Landmark building, the other main character of the effect was the giant Tiffany Diamond. Especially with the large scale of the diamond, we had to make sure the diamond was extra shiny and dimensional. We first started with an old diamond matcap we had previously made for a Rihanna filter and tested with that. But the Tiffany diamond had a really unique yellow color and a much higher density composition, so we ended up creating a custom matcap - using Midjourney and Photoshop.

To make the diamond even more magical, we added kira and bokeh effects as the diamond rotated. Caustics and pulsing glow were also layered on top to spotlight the diamond even more. The same kira shader was used for the bejeweled animals later.

Diamond Matcap

Animal Animations

If you open up the project, you'll notice that there were two different animation methods used for the bejeweled animals.

The bird and fish assets had fully baked animations, including their flight/swimming paths. Slight adjustments were made in Spark to change their transforms slightly and to offset their timing.

The jellyfish and seahorse assets had baked looping (in place) animations, but their transforms were all animated manually using Spark's keyframe patches. The keyframe workflow in Spark is still very nascent so I would definitely recommend avoiding it as much as possible. Although doable, the process was very tedious and not efficient at all. If we do ever need to use the keyframe patches in the future, I'd recommend saving it for really simple and linear animations and/or having CG bake out the complete animation and import it as a temporary visual aid for animating it in Spark.

We decided to keyframe in Spark largely because we were tight on file size budget (this was before client gave us 10MB).

Environment Textures and Shadow Mapping

Aside from more simple surfaces like the ground, pavement and the street sign, pretty much everything in the scene is using a baked color and normal map. This was necessary to match our style refs completely. We also had to use baked shadow maps for the buildings since Spark isn't able to dynamically cast shadows yet.

Final Demo

Back-camera demo

Front-camera demo

Year: 2022
Client: Meta, Fendi
Role: Lead Creative Technologist, AR Developer