Natural Language Computing

Antonello Crimi, Jess Leitch, and Jason Severs write about natural language acting as the next generation of user experience:

This transformation is exemplified by the Rabbit r1, an innovative personal AI assistant. Its Large Action Model (LAM), a type of “universal controller for apps,” not only learns your preferences, but performs actions aligned with them: anticipating your needs and seamlessly orchestrating tasks across various apps. Imagine planning a weekend getaway. Once trained on your preferences, the r1 could manage everything—from flights and hotels to restaurant reservations and activity bookings—crafting a seamless tapestry of experiences rather than a disjointed series of app interactions. Or train it to streamline paying bills, avoiding the need for dealing with a phone tree or unresponsive online forms.

It’s fair to believe that a possible future of interacting with computers is driven more by natural language queries that can actually achieve results instead of requiring users to click or tap dozens of times. The question I have is how will these systems convey complex outputs and confirmations? I strongly believe a role for user experience designers in this future is to ensure that important information like costs, flight schedules, hotel rooms, and other purchases are communicated to users with clarity, familiarity, and branding.

For example, if I want to book a suite at the Four Seasons in Philadelphia between May 3 and May 6 with a massage on Saturday at 2pm and a late checkout, how will an AI-based interface express that information for me to review before confirming? As of today it could look something like this:

Item	Selection
Hotel	Four Seasons
City	Philadelphia, PA
Room type	Suite
Dates	Friday, May 3 - Monday, May 6
Massage	Saturday, May 4 at 2:00PM
Late checkout	Monday, May 6 at 4:00PM

This is certainly an efficient way to convey a lot of information, but it’s awfully drab considering I am trying to book a vacation. Today when using the Four Seasons website I get photos of the room and hotel, information about the spa, rates for different sets of days, options to upgrade, the ability to use loyalty points, etc. I think the convenience is welcome, but I am losing a lot of the experience as a result.

Fortunately this can also be solved with AI. We must seek to build interfaces that can convey complexity in a way that humans actually enjoy. AI should be expected to render beautiful, enticing pages instantly when given a set of data. Querying a hotel’s website, retrieving options, and displaying results in a bulleted list is not the end of the AI’s task. Imagine if the apps we used today just displayed API calls or giant blocks of text. As designers we bring elements like empathy, elegance, tone, and consistency to ensure users can complete tasks. The same applies to a world where we interact with AI instead of apps.

The future is not text; it’s everything the user needs to see in the moment to confidently make a decision.

AI Apps