One takeaway from this year’s WWDC had nothing to do with what was announced on the virtual stage. Instead, it was found in the (virtual) audience.

Apple kicked off this year's WWDC keynote with a surprising twist: Tim Cook walked out on stage inside Steve Jobs Theater to an auditorium filled with 600 Memoji. (Yes, I tried to count every Memoji.)

While the decision to include that opening scene may come off as just a way to add some fun to the keynote, my suspicion is it was part of a much larger, multi-year bet Apple is placing that amounts to using Memoji to prepare us for the upcoming mixed reality era.

What Is a Memoji?

The straightforward answer is that a Memoji is a digital representation of how we see ourselves. This explains why Apple launched Memoji with significant customization options and has continued to add seemingly every minor facial and head accessory tweak possible. Apple doesn’t want anyone to feel left out or unrepresented.

Born in the pre-AR era, Memoji were introduced in 2018 alongside iOS 12. One year prior, Apple had introduced Animoji (animated emoji) as a way of utilizing the iPhone X’s TrueDepth camera system.

The process used to create a Memoji provides clues as to how Apple sees them being used. It’s all about communication. One can navigate to the Messages app to create a Memoji. At the end of the Memoji creation process, sticker packs are offered to add more personality and touch to messages. Last year, Apple unveiled a Memoji app for Apple Watch allowing Memoji to be created on the wrist. Creating a Memoji is surprisingly fun, easy, and even relaxing. After just five minutes of customization, here is my Memoji:

One thing that caught my attention with Memoji is how they have a distinct look. It’s easy to pick out a Memoji from various other digital representations of oneself available via a growing number of third-party apps. In an interesting way, Memoji ends up being a form of Apple branding next to the Apple Watch’s distinctive rectangular watch face, the front-facing camera and sensor module on iPhone, and the iMac front-facing chin.

WWDC 2021

Apple continues to lay the groundwork for a move into mixed reality. The clues of this transition were found throughout this year’s WWDC keynote.

FaceTime SharePlay will have a big role to play in mixed reality as we consume content while simultaneously interacting with friends and family.
Live Text in Photos is an obvious feature destined to be used while wearing Apple face wearables.
Apple Maps received additional AR features that are perfectly suited for smart glasses.
Spatial audio is all about rethinking the way we consume (audio) content depending on our relationship between the physical and digital worlds.

There is plenty to talk about with each of the preceding items. Some of that discussion was found in my WWDC keynote review available here. However, there was one clue supportive of Apple’s transition to mixed reality that was not included in the preceding list or in anyone else’s “WWDC clues pointing to mixed reality” list: a heavy emphasis on Memoji.

Apple went all out with Memoji at this year’s virtual WWDC. We got an early hint of this with the WWDC keynote invites. As shown below, the invites included three Memoji looking at MacBooks. Last year’s WWDC keynote invite, also shown below, was similar.

Some may look at the invites as merely reflections of Apple hosting virtual WWDCs. However, that explanation is unsatisfying. The MacBooks being looked at are the references to WWDC going virtual due to the pandemic, not the Memoji themselves.

In addition, Apple updated its leadership page, shown below, to include everyone’s Memoji.

While revising the leadership page wasn’t unprecedented as Apple did something similar to mark world emoji day in 2018, comparing the two pages shows how Apple has been serious in improving Memoji. Apple SVPs also had their Twitter profile pictures converted to Memoji.

There was then the WWDC keynote’s opening scene, which could have benefited from a bit more commentary. Due to the pandemic, Steve Jobs Theater, Apple’s $100+ million 1,000-seat underground theater, has not hosted a product event since the second half of 2019. While the theater was shown in prior virtual events, there was something about having Tim Cook stand in front of an empty theater that just didn’t feel right. By including Memoji, Apple was able to add some life, albeit animated, to the theater.

Mixed Reality

As for why Apple’s Memoji push stands out to me, Memoji is a tool Apple is relying on to prepare users for mixed realty and completely rethought ways to consume content and communicate with others.

Most people are familiar with the terms augmented reality and virtual reality. However, such terms have become confusing when figuring out what they actually mean or describe. “Mixed reality” is a more encompassing term that simply refers to technology made available via comfortable head-worn visors and goggles. Instead of being worn all day, such devices would be designed to be used while seated.

Mixed reality will introduce the idea of sharing real-world experiences with others via a digital space. Examples include “attending” everything from live sports events and awards shows to theatrical plays from the comfort of one’s home. Given the right camera technology, any real-world event will likely one day be able to be consumed in mixed reality. Instead of these live events occurring in a digital world with a vibe similar to that of a video game like The Sims or Fortnite, the user will feel like they are actually attending the event seated in the front row at Madison Square Garden or the Staples Center.

Given how live events are all about communal experiences with friends and family, mixed reality has to be able to replicate shared experiences. This will require a method of representing oneself to others while “attending” the live event via mixed reality. Instead of using odd-looking avatars or actual portraits of ourselves that have been animated to appear life-like, Memoji is Apple’s answer for that digital representation.

By pushing Memoji now, prior to actually selling mixed reality devices to the public, Apple is looking to remove whatever awkwardness may be found with creating and using a digital representation of ourselves. Higher Memoji adoption will then make it easier for people to embrace mixed reality when it is time. Strong adoption for mixed reality would then help Apple’s efforts to develop a platform developers can use to rethink our communication and social activities.

Apple M&A Clues

Apple recently acquired two companies that were involved in sharing real-world experiences. NextVR played in the realm of using cameras to map the real world, creating a “virtual world that is painted with images of the real world.” The company had also begun work on placing elements of live experiences “into the viewer’s physical world,” to use NextVR’s own description.

These "virtual" experiences can be achieved by mapping indoor sports arenas, concert halls, or practically any room for that matter. The point in building indoor maps isn’t to take the user into some kind of imaginary world similar to Mark Zuckerberg’s Metaverse. Instead, it’s all about delivering enhanced experiences for consuming content. Although it’s difficult to portray this point using traditional video, the following example from the 2017 Wimbledon final (via NextVR) provides some ideas as to the possibilities.

Simply put, there is something compelling found there. Apple could take this technology and interlace it with Memoji to offer a new method of consuming live sporting events digitally with friends and family.

Last year, Apple also acquired Spaces, which was founded and incubated within DreamWorks Animation. The Spaces team has an interesting perspective to share in the entertainment space. The leadership team view themselves as game developers – which explains the company’s stated mission of focusing on experiences. Spaces worked with the National Geographic Society in offering an audience the ability to consume the same VR experience led by a presenter.

About the Metaverse

As for Zuckerberg’s vision involving the Metaverse in which people will spend time in a digital world instead of hopping between digital and the real life, the idea is flawed for one very simple reason. The Metaverse is at odds with a trend that has been unfolding for decades and will continue to unfold for decades to come. As technology has become more personal, the barriers between humans and technology have eroded. It’s hard to think of a larger barrier between technology and humans than what is being described as the Metaverse.

The value and promise found with mixed reality isn’t to remove us from the real world but to bring the real world to us. There will always be a limited number of front row seats available at a sports arena or concert hall. By allowing a sports game or concert to be consumed in mixed reality, everyone will be able to have that front row experience. Instead of merely watching Tim Cook walk out on stage to kick off a product event in Steve Jobs Theater, we will be able to experience what it’s like to sit inside Steve Jobs Theater.

Listen to the corresponding Above Avalon podcast episode (24 minutes) for this article here.

Receive my analysis and perspective on Apple throughout the week via exclusive daily updates (3 stories per day, 12 stories per week). Available to Above Avalon members. To sign up and for more information on membership, visit the membership page.

For additional discussion on this topic, check out the Above Avalon daily update from August 24th.