Apple and Artificial Intelligence

WWDC 2024 is approaching, and we all assume Apple will share how AI will impact their hardware and software. Expectations are incredibly high. I thought it would be a fun exercise to think through a variety of approaches that Apple can take based on what we have seen from Google, Microsoft, OpenAI, and others.

It’s important to note that while Apple may be behind in incorporating AI into its various operating systems, they have used machine learning for several years in a few crucial areas. For example, detecting faces in photos in iOS 11 and using a transformer to improve autocorrect while typing in iOS 17.

The big questions for the next phase of Apple’s AI efforts are:

1. What should AI do for users?
2. What should AI feel like for users?
3. How can developers leverage Apple-provided AI?

Option 1: A Better Siri

Similar to Apple claiming autocorrect would be more reliable in iOS 17 by using a transformer, Apple could similarly claim that Siri is now better without making any changes to its interface in any operating system. Users know how to invoke Siri, they have a decent idea of what its capabilities are, and expectations are low. If Apple can increase accuracy and consistently return results better than “here’s what I found on the web,” this could be a win. In this case Siri would also become conversational by allowing users to ask follow-up questions which is now expected behavior based on competing products like ChatGPT.

The Better Siri approach is extremely risky though. Apple will be perceived as behind for another year as Google and Microsoft continue to expand their AI offerings with new interfaces and capabilities while Apple’s AI will be trapped inside of Siri. Google already released Gemini as a standalone product, AI-powered search results, new generative text features in Google Workspace, and a growing list of AI features specific to Android. Microsoft, thanks to its partnership with OpenAI, is also moving extremely fast with an AI-powered Bing, Copilot in Windows, and Copilot in Office (Microsoft 365).

This approach could expand what users can do with Siri, but I’m afraid without substantial changes to the interface it will not change how users feel about using Siri. It also may not get developers excited to reinvest in SiriKit if their customers continue to have a generally negative outlook on Siri.

Option 2: A New Destination

Siri has always existed on the periphery. You invoke it, get a snippet of information (or quickly take action), and leave. Users do not stay in Siri long enough to be productive, develop ideas, or complete complex tasks. This can certainly change. Siri can transform into a destination with permanence. Perhaps Apple will release a new Siri app users can launch, interact with for more than a few seconds, and return to at a later time to continue working.

A more likely direction for an AI destination is to replace a core home screen interaction like swiping left to right to access a more advanced Siri interface (and remove the redundant widget screen). This would feel more connected to the OS as a part of SpringBoard vs. an app that can be moved or deleted.

Would users see this new interface if they said “Siri” or held down the power button? Siri already has the ability to complete quick tasks, ask clarifying questions, and show confirmations without taking over the screen. Moving from the temporary, partial screen state to a full screen state seems like a step backwards. I like how Siri currently only covers the necessary pixels to accomplish a task. For example, today I can say “Siri remind me to write a blog post later” and I only see the temporary Siri animation followed by a Reminders confirmation component. What would be gained by going full screen here?

If Apple did release a Siri app or a more permanent experience what would it actually do? Would it feel conversational? Would it allow you to view prior queries, actions, and confirmations? Would developers have the ability to integrate with it? Would it preemptively collect and display information you didn’t know you needed to see? Surfacing helpful information already exists in several ways. For example, when you enter the search interface on iOS “Recent Searches” may appear, or when you park a car that was using CarPlay a notification appears to remind you that your parking location was stored. Do we need more of this in a centralized location? Also, what would an AI destination look like on macOS and watchOS? Would macOS have a new app in the dock by default in the next major release? Clearly many questions need answering, and a designer could explore concepts forever. However, I do not believe a destination is the direction Apple will take for AI because it should be accessible everywhere; not confined.

Apple does occasionally release new apps, but they always have a very clear purpose. Journal is for documenting your life. Clips is for making fun videos. Podcasts, Music, Books, etc. A Siri app is for… talking to Apple’s AI? Why would I use this app over ChatGPT? Perhaps Apple’s conversational, LLM-powered app allows me to interact with the vast amount of personal data Apple has access to: calendars, contacts, email, browsing history, iMessages, photos, etc. Maybe Greplin Cue is coming back!

Option 3: A New Layer

A theme across the majority of recently launched AI products is generative text. For example in Gmail I can ask AI to help me draft an email. Once I have a draft I can further augment it using AI by selecting options like “formalize,” “elaborate,” and “shorten.” I assume more freeform options like “make it fun” are coming. We’ve already seen this in Humane’s demo video, What is Ai Pin. In the video Bethany Bongiorno, Humane’s cofounder, asks AI to make her message “sound like Gen Z” (oy). Oh and if you own a Pixel 8 or Galaxy S24 you can use Magic Compose to draft text messages on device thanks to Gemini Nano. Yes, this is going to all be confusing for a few years. That is why we need Apple to package it in a consumable manner.

I believe generating text is both feasible and the key to Apple catching up to Google and Microsoft on the consumer side. Wherever the user has a blinking cursor users should be able to invoke Siri and speak a few words to receive help with writing text. This addresses the question of what users can do with Apple’s AI, and it will feel exciting because AI will now be available everywhere vs. stuck in an app or website. Instead of launching the ChatGPT app, composing a few prompts to achieve a satisfying result, copying text, launching another app, and then finally pasting text, users can interact with AI instantly. This will also introduce AI in a consumer-friendly way to potentially hundreds of millions of people.

For developers I assume there will be an opportunity to offer up data or functionality that Siri can access as a way to contribute to users’ queries. For example, today in ChatGPT if I ask, “Can you get URLs to Wikipedia for each Mac that launched in 1995,” I do not actually get a list of URLs. Instead I get a list of Macs that launched in 1995 (the Power Mac 9500 and PowerBook 5300) and a link to “List of Mac models” on Wikipedia which includes all models. I consider this a failure. If I’m in iOS and I have the Wikipedia app installed, perhaps there will be a way to reliably respond to this query using an LLM-powered Siri.

A more exciting scenario (and a bit more difficult to believe is possible) is accomplishing complex tasks using Siri. Imagine I launch Things, my favorite tasks app, with the goal of creating tasks to prepare for all of tomorrow’s meetings. I say, “Siri make a task for each event I have tomorrow.” Things can now ingest my calendar data, make an array of events I have scheduled tomorrow, and then create a list of tasks populated by the event array. This is now starting to sound like a supercharged Spotlight in addition to providing generative text.

The New Layer direction is sound because it expands Siri’s capabilities for both users and developers without making large changes to each OS. Users constantly see blinking cursors, they know how to invoke Siri, and with the power of an LLM they can (hopefully) speak naturally with satisfying results. The New Layer meets users where they are: in creation mode. While actively writing I will have a way to ask for help. For the interface I assume there will be both a confirmation step to insert the new text, and a way to augment it with an additional command similar to Gmail’s functionality discussed above.

The Siri Brand

People who think Apple will rebrand Siri have not clearly studied Apple’s history, nor have they worked in branding before. The cost to rebrand is exorbitant and will cause confusion for years. Imagine Apple supporting two words to invoke an assistant! Eventually they would remove one? Or imagine announcing “Siri” stops working when new versions of iOS, watchOS, macOS, tvOS, and visionOS launch later this year and users are expected to immediately learn the new word.

I agree with the general consensus that the brand is not particularly well-received, but it is strong and familiar. My guess is people hear “Siri” and think of timers, a confusing voice they occasionally hear from a watch or phone, or a thing Apple makes that they tried many, many years ago. They do not think intelligent, reliable, fun, helpful, etc. I asked a few people who are not in tech “What do you think of when I say the word ‘Siri?’” Here are their responses:

• 40s, female, marketing executive: “I think of something that does not work.”
• 70s, female, interior designer: “Annoying. I do not like it.”
• Teens, female, high school student: “My phone and Apple.”
• Teens, male, college student: “Semi helpful.”
• 30s, female, sales leader: “She’s totally incompetent.”
• 30s, female, merchandising executive: “She is dumb.”
• 70s, female, retired EA: “She sets my alarm.”
• 30s, male, banker: “Annoying.”
• 30s, male, environmental engineer: “Don’t use it.”

Similar to Apple Maps’ ability to win over users, there is an opportunity for Siri to grow.

Of course Apple has rebranded a few products so there is precedent: Apple Computer became Apple Inc. (2007), Mac OS X became macOS (2016), iTunes became Music (2019), iTools became .Mac (2002) which became MobileMe (2008) which became iCloud (2011), and iPhoto became Photos (2015). Apple is a different company that it was even 5 years ago when Music launched. As a result a rebrand seems very difficult to imagine. A more likely change is the introduction of a paid tier of Siri like Siri+. Perhaps for $5 per month you gain access to an LLM-powered version of Siri across your devices.

My Dream for Apple

Imran Chaudhri, Humane’s other cofounder, explains his hypothesis for the future of compute in Ai Pin Explained. He believes presence and freedom are key themes. In other words, users should have access to infinite data and functionality without constantly looking at a screen. I think this is a possibility, but not necessarily in this decade. Another possibility is we become even more dependent on our phones and computers because of AI.

If we imagine a world with infinite compute, I can have a personalized AI that is trained on all of my data. Everything. Every document I’ve written, message I’ve sent, photo I’ve taken, etc. Only Google, Apple, and maybe Meta can achieve this through their operating systems, apps, and services that we love or heavily rely on. Imagine instead of interacting with Siri I could interact with myself. Based on everything I’ve ever done with a computer, what would I write or click on next? Perhaps creating a Persona with the Vision Pro is step 1, and step 10 is imbuing my Persona with an LLM that is… me.

Apple Watch Faces

For years I have felt a tension across iOS, macOS, and watchOS between simple, subtle interfaces and vibrant, complex interfaces. We have beautiful, high pixel density screens surrounding us all day every day. Should they be used to their full potential through the cramming of widgets, windows, and complications? Or should they exist on the periphery, whispering a minimal amount of data?

Since dark mode first appeared in macOS Mojave in 2018 and iOS 13 in 2019 I was hooked. I prioritized apps that supported dark mode, customized dock and app icons to be dark, and even used plugins to make some web apps dark. Emphasizing blacks and grays pushed me to become a computer minimalist, reducing the amount of toolbars and icons to let content stand out. Who needs icons and buttons when one can just memorize keyboard commands anyway. One could argue I took this a bit far (as I usually do with computing trends).

With iOS this dark mode minimalism manifested by only using 1 screen of apps, and reducing the number of app icons on the Home Screen to just 8 alongside small calendar and task widgets leaving plenty of unused space. My wallpaper and lock screen were either all black are a subtle gradient from dark gray to black depending on my mood. On macOS I meticulously chose which apps earned a place in the dock to ensure it was always as thin as possible. My wallpaper was randomly selected by Unsplash with an emphasis on dark, minimalist, architectural photos.

Over time I felt like beauty and joy were missing from my digital life. What if I allowed a spot of color in a few places? What if I… turned off dark mode? Years had passed since I even tried light mode. I flipped the switch on my iPhone and was instantly reminded of how computers are supposed to look. Light! I felt reconnected and rejuvenated. My devices felt fun again. iOS and macOS were reborn.

The one device where I continued to struggle was the Apple Watch. Since its screen is always on it constantly draws attention. My kids’ eyes find their way to it for no reason while talking or plying. When weighing the interface’s beauty vs. its ability to distract, I ultimately believe its vibrancy and complexity should be reduced. The watch should not shout “look at me!” It should exist on the periphery. It should be patient.

When I access the watch I should not be distracted from my primary thought process. I need to quickly and subconsciously glance, ascertain the information I am seeking, and gracefully return to be present with my task or conversation. As a result I transitioned from Modular Ultra and Wayfinder to Metropolitan. It’s elegant, and I find the elongated clock digits align with my enjoyment of crisp graphic design. It’s the face I think Massimo and Lella Vignelli would choose.

Deer Valley

Earlier this month Remy and I traveled to Park City for the annual Binswanger Cousins event. The tradition started with her grandparents twelve years ago (and coincidentally one of the trips is where my sister in law met her husband). An important thing to know about the Binswangers is they take partying very seriously. Costumes, games, ceremonies, and plenty of laughter.

Notably the group held a small pseudo-religious service to celebrate the 2 year anniversary of my horrific accident on that mountain. I shattered my tibial plateau while skiing down Big Stick with a 10 week old baby expecting me to be a functioning parent. We all went to the approximate spot of where I fell and secretly hung up a token on a tree. I can’t wait to revisit it next year.

As the photography enthusiast I took on the responsibility of documenting the shenanigans using my dinner party camera: the Leica Sofort 2. I learned a few things about the Sofort after using it for several hours:

1. People love seeing printed photos in real time. Holding them, photographing them, rearranging them, etc. It was a huge hit.
2. The camera has very limited storage, and you cannot take photos when the storage is full. I found myself constantly sneaking away from the festivities to manually transfer photos to my iPhone so I could delete them from the Sofort. One way to remedy this is to insert a micro SD card to provide additional storage. This gives you a lot more flexibility (and allows you to enjoy the fun).
3. Overall it felt unreliable. Photos would occasionally appear blurry and I was unable to figure a cause or pattern.
4. It’s slow. Operating it, transferring photos, using the menu, etc. Slow.

The next step is figuring out how to make a collage with all the tiny prints.

AppleDesign Book

I decided to expand my Apple Collection (“Museum” and “Collection” are interchangeable at this point) to include books in addition to desktops, laptop, peripherals, displays, etc. I found a copy of AppleDesign: The Work of the Apple Industrial Design Group. It’s an incredible book filled with gorgeous photos of products and prototypes.

40th Anniversary of the Macintosh

Yesterday was the 40th anniversary of the Apple Macintosh’s launch. I recommend watching a few minutes of Steve Jobs introducing the Mac at De Anza College in Cupertino, CA on YouTube. While watching try to imagine a world where computers didn’t have a graphical user interface; just a blinking cursor begging you to enter commands. The Chariots of Fire theme song feels cheesy when watching in 2024, but in the moment I can imagine it felt appropriate.

The Steve Jobs Archive wrote a nice piece called “40 years of the Macintosh” for the occasion. One paragraph stuck out:

“I remember the week before we launched the Mac,” Steve recalled in 2007. “We all got together, and we said, ‘Every computer is going to work this way. You can’t argue about that anymore. You can argue about how long it will take, but you can’t argue about it anymore.’”

This is how I always felt when I was growing up. In the early 1990s (and to be honest through college) I argued with kids and adults about why Apple computers were superior machines. Yes, superior. One could argue I was a bit pompous as a child during these conversations, but I was so passionate about Apple. Windows machines could always do more, but the software was ugly and the hardware was clunky.

While visiting friends’ houses we would sit in front of Macs for hours and just figure out what they could do (in addition to playing Maxis games). We didn’t have modems yet so we were forced to poke around the system with apps like ResEdit. Macs were approachable. Elegant. My friends and I never tried to tinker with a PC. They were built for parents and work. Macs were for us.

The Upgrade podcast collected some of my favorite Apple enthusiasts to discuss the anniversary along with their first Mac, favorite Mac, favorite Mac software, favorite Mac accessory, and their least favorite Mac (the hall of shame). I put together a list as well.

First Mac

My first Mac was the Mac LC with a 16 Mhz 68020 processor, 40 MB hard drive, and 2 MB of RAM. This is where it all began for me. I remember listening to the national anthems of every country on it and feeling amazed that I had access to so much information. Playing with Kid Pix made me feel like an artist. Perhaps I should find one for my collection.

Favorite Mac

My favorite Mac is the Power Mac G3 Blue and White with a 400 Mhz G3 processor, 6 GB hard drive, and 64 MB of RAM. I had it configured with a built-in Zip drive too, and over the years I filled all the RAM slots and added a couple extra hard drives inside. This computer was a beast. It acted as a bridge between the Classic and OS X eras which was a particularly exciting time in Apple’s history. My parents were kind enough to ship it to New York for my freshman year of college where I quickly learned that a tower and CRT monitor combination was not suitable for a tiny dorm room. The best part? I still have it.

Favorite Mac software

My favorite piece of software is Marathon. Hours and hours were joyfully spent playing this game against friends at Crystal Springs summer “school.” I was fortunate to spend a few summers in front of Power Macs learning how to create animations in Macromedia Director and rendering 3D scenes in Strata Studio Pro (when we weren’t playing Marathon of course).

Favorite Mac accessory

My favorite Mac accessory has to be the Airport Express. It was the perfect product for a college student, and I was a senior when it launched. We had laptops, speakers, and giant mp3 collections. The problem was figuring out how to play music wirelessly from any computer in the house or apartment. The Airport Express arrived and everyone could easily play music from iTunes without disconnecting their laptop, moving it to the living room, and reconnecting. It was also a fun party trick! In 2004 no one even imagined wirelessly playing music. It was seamless. It was cool. It, dare I say, just worked.

Hall of Shame

This category is tough. Does one select an unreliable model? Or a model that had a problematic keyboard? My gut tells me the Hall of Shame should be awarded to the Power Mac G4 Cube. Let’s be very clear: the Cube, its speakers, and its matching Cinema Display are GORGEOUS. Cramming the computer into an 8 inch cube was extremely impressive. However, I have some issues.

When this computer launched I was selling Macs at a local third party Apple retail store. The Cube was set up right by the front door. Everyone was clearly impressed by its design, but I don’t recall ever selling one. One person came to the store to complain that the capacitive power button on the top was too easy to accidentally press. He would randomly turn off the computer! Then there were the cracks. The computer didn’t have a fan and its healing solution was unable to adequately cool the acryclic glass enclosure. As a result it was likely to eventually crack. At $1,799 (with a $499 display) it was a tough sell considering its expandability limitations and low performance compared to its G4 tower sibling. This may all sound like nitpicking, but the cracking exterior is unforgiveable.

Stash

I adore Minimalissimo and recently saw that they added a new Stream page to the website. Introducing the Stream:

But what if you come to Minimalissimo to get inspired? What if you have an appreciation for minimal design and you simply want to immerse yourself in the content? What if the goal is not to search anything specifically, but rather to stumble on something you didn’t know you were looking for?

I have always wanted a way to quickly grab images or screenshots, drop them into a folder, and have them appear on this website. No text. No tags. No descriptions. Just stashing away things that I think are cool. When Minimalissimo shared this update I was inspired to finally add it here, but with a name that resonates with me: Stash. Enjoy!