Google I/O Puts Focus on Speedy, Accessible New Interfaces

At this year’s I/O conference, Google unveiled several new features related to its upcoming Android release and devices. Among the most exciting of these features are those that aim to change the dominant interface through which users engage with their devices: typing on a keyboard. While the touch screen revolutionized media about a decade ago, it looks like the camera and microphone are ready to take the baton, at least when it comes to accessing on-the-go info.

Google CEO Sundar Pichai’s keynote event was brimming with new features and products that are set to change how we interact with devices and each other in everyday situations. We paid tribute to this spirit of progress by producing an animated countdown video that kicked off the keynote, taking viewers on a journey through advances in tech over the decades.

Many of Google’s most interesting announcements centered on voice and visual search in particular, making the case that these new features could provide its users with information much faster than if they had to type it out. While this has always been the idea with voice interfaces, this year’s I/O event delivers on the promise by showcasing instantaneous voice recognition with its Assistant and surfacing up actionable information through augmented reality. Here are the feature’s we’re most excited about—and where the value lies for users and brands alike.

Google Assistant Becomes More Human

Google announced several improvements to its Assistant, many of which center around on-device voice recognition. Previously, Google’s voice recognition model was 100GB, requiring queries to connect to the cloud before getting a response. Now, Google has managed to shrink that model down to an impressive half-gigabyte, small enough to fit on devices for rapid, offline voice recognition.

“Now that Google can do recognition on the device itself, the device can actively listen and respond without the need to go through the loop of saying ‘Hey Google’ followed by a command,” says Michiel Brinkers, Technical Director at MediaMonks. “You can simply keep talking to the Assistant with follow-up questions.” It also allows for contextual commands, such as saying “Stop” to stop media playback on the device, no “Ok Google” required.

Google’s newly unveiled Nest Hub Max device—which is a mix between the Home Hub and Nest camera—even adds physical gestures to the interface. Thanks to facial recognition, the Nest Hub Max can alert users when it notices someone in the home it doesn’t recognize, or greet them with personalized content when they’re in view. The latter solves a crucial problem faced by Internet of Things devices: when multiple users in a home share a single device, how do you target them individually with personalized content? We’re excited to see Google crack the case while alleviating privacy concerns with on-device facial recognition.

Speedy Voice Recognition Will Change Users’ Lives

The greatly improved speed achieved through offline Assistant interactions is a game-changer on mobile devices, where wait times or lack of connection can be a huge pain point. “If a voice assistant doesn’t instantly do what you want it to, or if it gets it wrong, then it becomes more effort to use that system than to accomplish the task through typing or tapping,” says Brinkers. “But what Google showed offers a huge improvement.”

While on-device voice recognition can make many of our lives easier, but for some it will be life-changing: thanks to immediate transcriptions, Android devices will be able to provide users with auto-generated subtitles for any video or audio (including live content), an obvious benefit to the hard-of-hearing. In addition, Google announced its Project Euphonia program, which will provide larger data sets to train the Assistant to better understand those with speech impairments. “These initiatives are where a company like Google shows their value to the greater good,” says Brinkers.

As a Technical Director, the faster, improved speech recognition turns the creative wheels in Brinkers’ mind. “If voice does become a dominant input method, maybe we can listen to tone of voice—do more than just listen to what’s being said, but how it’s said,” he muses. “Then we could identify their emotion and design experiences around that.”

Google Lens Brings Printed & Digital Content Together

Voice isn’t the only interface Google is gunning for this year: the company also revealed several new AR features. While most consumers’ experience with AR has been focused explicitly on entertainment, I/O demonstrated how much the technology has matured in the past year to provide users with actionable, contextual information they can use in their daily lives.

One example shown in Google’s keynote is the ability to scan a restaurant menu with a phone using Google Lens. Doing so provides users with a list of a restaurant’s most popular dishes, reviews and photos. We’ve long said that the camera is the new browser, and new Lens features offer a textbook example of what that future could truly look like. “If I could read any restaurant menu in a foreign country and see what the food looks like through my phone, that would be amazing,” says Brinkers. “We always ask ourselves what the utility use case is for AR. This is it.”

In addition to providing greater contextual information, Google showcased Lens’ ability to animate traditional, static media—one of the coolest features for those who always wished they could read an animated newspaper as seen in the Harry Potter universe. One example demoed at the event is a poster depicting the Eiffel Tower. When scanned with Google Lens, clouds in the sky begin to move, bringing the image to life.

The tech isn’t just about cool visual effects, though—it also has utility, particularly with how-to content. Scan a recipe in a magazine with Lens, and a video tutorial can overlay atop it to show how the dish is prepared. What really places Lens at the forefront of AR is that the scanned media doesn’t require abstract, distracting markers or QR codes to activate; the content itself is the key, enabling a more elegant way to augment printed media.

Get Up-Close & Personal with Google Search Results Using AR

Later this year, users will find 3D models in Google search results, allowing them to examine the object or thing they’re searching for from any angle. If that’s not already cool enough, Google is upping the ante by letting users place the object in front of them using AR. This functionality offers a simple, intuitive way for users to learn about real-world objects and preview products.

“If you searched a chair on Google, it would be neat to drop it down in your room and see how it looks,” says Brinkers. “It will be interesting to see how this competes with proprietary apps that already let you do something similar.” One benefit that searchable AR objects have over those native apps is that users can view them without having to download and install anything. Google is exploring brand partnerships for developing these models in search, signaling the potential value it can have for marketing.

What’s truly exciting about each of these developments is their potential to come together in one unified experience. Scan a sign in a foreign language with Lens, for example, and Google can verbally read it back to you in your own language through advanced text-to-speech. Marry visual and voice features with an augmented reality layer, and the way we interact with everyday devices—if not the environments around us—may radically alter in the next couple of years. “What’s interesting with this event is that you see a lot of synergy between AR and machine learning,” says Brinkers. “Google is combining all these tools that they’ve worked on separately, and we see it coming together in a way that no one anticipated.” Ok Google, what’ll it be next?