Back

#best practices

#ux audit

8 min

Human vs. AI: Who’s Better at UX Design? You Know the Answer

We asked AI to do our UX design job for us. Here’s what we got

Katrin Shabina

Chief Design Officer

Ellen Snesar

Content Manager

October 11, 2024

Intro Level: easy Level: medium Level: hard Key takeaways

In our 12 years as a digital agency, we’ve faced many challenges in designing apps and websites. Until recently, we tackled these challenges on our own, without the aid of artificial intelligence.

But in 2024, when AI became so ubiquitous, we wondered: how would AI handle the tasks we had? Would it come up with the same ideas we did? To find out, we asked ChatGPT to tackle some of our past tasks and compared its solutions to ours. Here’s what we discovered.

Level: easy

Find ideas for an empty state illustration

Empty state screens, like when nothing is found or a cart is empty, are common in apps. However, our best practice is to never leave these screens truly empty; instead, we add a unique touch where relevant.

For a super app we developed for a sporting goods retailer, we wanted the empty states to feature illustrations that aligned with the client’s brand.

AI’s result. To see what AI would do in this situation, we asked ChatGPT to suggest illustrations for an empty cart. The AI came up with ideas like a basketball hoop with a ball “just missing,” a cyclist on a road, and an “adventurous” backpack.

Interestingly, it also suggested copy, such as “Score big with our gear!” for the basketball hoop. We visualized these ideas using Microsoft Designer:

View the original ChatGPT chat

In our prompt, we didn’t specify colors, just the overall style of the illustrations—isometric perspective, simple shapes, clean and uncluttered look.

Human’s result. Unlike ChatGPT, we went beyond the sporting goods theme and drew inspiration from the whole product range, including adventure gear and outdoor clothing. Our illustrations were designed in a flat, minimalistic style to keep the focus on key interface elements.

We also subtly guided users to browsing the catalog, adding items to the cart, or liking a product.

“AI answers are a good starting point, and with more specific instructions, you could get even better results. However, I prefer our approach—it’s more nuanced. For example, ChatGPT didn’t suggest using a compass, which isn’t strictly a sports tool but fits our vision perfectly.”

Katrin

Chief Design Officer at HeyInnovations

Create an app mascot

Sometimes, an app needs more than just themed illustrations—it needs a mascot. If the client doesn’t already have one, it’s our job to create it. That’s what we did when designing an app for a milk and dairy producer.

We aimed to create a character that would guide users throughout the app, introducing features and sections during onboarding and through on-screen prompts.

AI’s result. We asked ChatGPT for mascot ideas and suggestions on where it would incorporate the mascot in an app. It provided us with four concepts:

Cool Cream the Ice Cream Cone,
Sprinkle the Ice Cream Fairy,
Scoopy the Penguin,
Moo the Cow.

According to AI, these mascots could be used in various places within the app—onboarding, tutorials, prompts, loading screens, notifications, error screens, reward screens, and settings. To visualize the ideas, we used DALL-E to help us imagine what a mascot might look like:

View the original ChatGPT chat

As before, we didn’t specify colors, focusing instead on the overall style of the image—minimalist and childlike graphic design, with a cartoonish, playful, and exaggerated appearance, using simple lines and shapes.

Human’s result. The final mascot created by our graphic designer back in 2021 is quite similar to ChatGPT’s Cool Cream the Ice Cream Cone, though we didn’t give it a name. Our version is a playful and friendly character who guides users through the app, explaining functions and sections during onboarding and prompts.

If something goes wrong, the mascot steps in to assist. For example, when the Internet connection drops, the ice cream cone suggests checking the connection, and when an error occurs, it recommends refreshing the screen.

“I loved that ChatGPT came up with names. It sparks creativity and helps visualize the character right away. However, even if AI generates something good, we’d still need to refine it—there are many factors to consider, like the client’s preferences, our own taste, and adapting images to different screens and sizes. With a generated image, that’s just not possible.”

Katrin

Chief Design Officer at HeyInnovations

Level: medium

Convey a scent through UX

E-commerce apps are designed to help users make decisions. This can be tricky with products like perfume, where physical characteristics are key. Our designers faced this challenge when building an app for a perfume subscription service.

Since users couldn’t try the perfumes in person, our task was to convey each scent through UX/UI elements. We wanted to make the online perfume shopping experience as easy and clear as testing scents in a store.

AI’s result. In our view, AI can help generate ideas for solving such tasks. We’ve checked that, and ChatGPT suggested some standard approaches, such as visual representations, preference surveys, and scent storytelling. However, it also proposed a fresh idea: using the sound of ocean waves to evoke the feel of an aquatic fragrance.

View the original ChatGPT chat

Human’s result. Our solution in 2018 was more straightforward. We used a note-by-note fragrance description, breaking down the scents into individual ingredients and representing them with bright illustrations and detailed product descriptions.

We also allowed users who owned the scent to describe it using filters and add their own descriptions to reviews. This helped other users better imagine the scent.

“Using sounds to evoke a scent, as suggested by AI, is intriguing. I’d love to use such an app. However, I’m unsure how this mechanism would work for complex fragrances with multiple notes.”

Katrin

Chief Design Officer at HeyInnovations

Devise additional services in the super app

We often work on super apps, which are mobile apps that combine a core service with additional ones. One such project was for a large bank, where we needed to implement cross-selling to enhance in-app sales and support the app’s ecosystem.

For example, we had to understand what additional services we could offer users who purchase movie tickets through the app.

AI’s result. We asked AI for ideas on services to add to the post-purchase screen. ChatGPT provided an extensive list, including ride-share booking, restaurant reservations, social media sharing, and calendar integration.

View the original ChatGPT chat

Human’s result. While AI can generate extensive lists of ideas, the key is always to focus on user needs and context. Even with a long list of suggestions, we need to prioritize and select only those most relevant to the app’s users.

Back in 2020, we decided to offer two additional services: booking a taxi ride to the movie theater and reserving restaurants nearby. The data we had on the bank’s customers showed us that those were the most needed services in that scenario.

View full case study

“ChatGPT version is more comprehensive, but it suggested adding 18 features to the ticket screen, which doesn’t make sense. You need to prioritize the features and understand which of them are really useful in this context. Only humans can do it.”

Katrin

Chief Design Officer at HeyInnovations

Level: hard

Improve e-commerce app UI

Sometimes, clients come to us with a mobile app that isn’t performing as expected, and we need to figure out why. This is why we conduct a digital audit first and then design a new app.

For instance, a sporting goods retailer we worked with had low app conversions, but the reasons were unclear. We first audited their app and then created a new mobile app design.

AI’s result. Can AI audit a mobile app? Turns out it can, but only to a limited extent. We uploaded a screen from the app to ChatGPT, and it identified nine UI issues. It also suggested some improvements, like enhancing visual hierarchy and text alignment.

At the same time, some suggestions were too general. For example, for AI, the navigation bar seemed “quite standard,” and the icons could be more “distinct and representative of their functions.” But its criteria wasn’t clear, given that we didnt’t explain the functions of any icons.

Human’s result. Unlike ChatGPT, we could use the app and analyze its performance with actual data. We identified four key issues:

Banners in varying styles led to banner blindness.
Small typography made product names and notifications hard to read.
Distracting ads and push notifications interrupted the shopping experience.
Brand tags replaced useful product information.

While ChatGPT offered some improvement suggestions, it couldn’t test or validate them. We could do that, so we redesigned the app with a new visual language, professional product photos, and updated typography to reflect different sports.

Our approach worked: user engagement increased, and banner click-through rates rose by 65%.

“ChatGPT’s suggestions, like ‘spin to win’ or a bright yellow button, need careful review. AI lacks the nuance to consider context and often proposes basic solutions that may not fit the client’s needs.”

Katrin

Chief Design Officer at HeyInnovations

Design a fitness ecosystem

Designing a digital ecosystem is one of our most complex tasks. It involves integrating various services across different platforms and requires not just a UX design process but also strategic thinking and a deep understanding of the client’s business.

For example, we had to create a wellness ecosystem for a fitness studio chain. The goal was to help young women stay fit, eat healthily, connect with others, and track their progress. The client wanted to support women beyond the studio and grow business with additional services.

AI’s result. ChatGPT suggested 10 services for the ecosystem, such as a calorie tracker and live fitness classes. It also advised how to prioritize launching the services, but without any specifics.

View the original ChatGPT chat

Human’s result. When we worked on this project in 2020, we started with user research to understand what features were truly needed. We also considered the client’s business goals. For instance, we added detailed trainer profiles, social features like ratings and games to boost engagement, and a clothing store to generate more revenue.

View full case study

“AI’s list is right, but it’s too general. How do we understand that users need a chat with a mental health specialist? Why would they not go search for an alternative? AI doesn’t know the answers to these basic questions. You need to talk to real users to understand their needs.”

Katrin

Chief Design Officer at HeyInnovations

Key takeaways

In UX design, the key word is UX—user experience. AI can neither use nor experience anything. It can assume what most people need, but that will still be different from what people actually need.

Also, AI’s ideas are too general. It’s looking for patterns and general insights, but interpretation is still on humans. It means AI’s answers are good as a starting point, but even if you add more details to the prompt, it won’t get specific.

In a nutshell, if we’d used AI while working on these projects, our creativity could’ve sparked faster. Still, finding great ideas and testing them would’ve been on us. That’s why an answer to the question “human or AI” in UX design is obvious—at least for the foreseeable future.

last update

October 11, 2024

Modular e-commerce apps for a retail giant with sub-brands