AI-powered UX done right
How LLMs can be used to significantly improve user experience in (almost) any app
Note: this post is inspired by the ideas described on Braindump.me. This team is working on “AI game studio”, and they show how they design and build LLM-powered UI. I definitely recommend to check it out: https://braindump.me/blog-posts/building-an-ai-game-studio.
LLMs have been with us for quite a while. ChatGPT or Claude have become irreplaceable helpers for many users when they search for something, want to understand how things work or need help with creating content. Lots of innovations continue happening in this area and we see AI-related news daily. There is a big trust among investors in the power of generative AI tools to improve productivity and even replace workers completely in some cases.
However, despite all of the LLM accomplishments, many people remain skeptical of their capabilities. It’s also common to make fun of companies which declare that they now “utilize AI” by integrating a ChatGPT-powered “ask a question” window to their software. For some, it seems that LLMs can strive as chatbots and autocomplete tools but nothing more.
It’s interesting that people in charge of software tech companies understand that chatbot is not the only way of utilizing LLM capabilities. For example, Notion CEO mentioned this in 2023: https://www.inc.com/jason-aten/ai-isnt-just-for-arguing-with-chatbots-why-notions-ceo-thinks-it-will-fundamentally-change-way-we-all-work.html
However, if we take a look at the ways different apps integrate AI, it turns out they still usually do it with chatbots, sometimes chatbots with a few predefined buttons and prompts. The same buttons as we have in any regular messenger chatbot. That’s it.
Here are some of these examples.
MS Copilot has a classical AI chatbot panel with a few generic buttons.
Notion has in-place “Ask AI” textbox that has predefined buttons. It's definitely better for some scenarios because it allows applying the AI action in the desired place and saves time for common actions. But still, I would argue that such company could do much better.
Why do I think so? Because recently I saw a very nice small example of “proper” AI usage by Google. And this is the company that usually doesn’t create the best UIs. Check out this Google Sheet recording:
Though very simple, it really demonstrates how AI-powered UX can be implemented to feel like magic. No prompting input field, no separate panels taking 20% of the screen. Just the app that understands your intention, asks for one confirmation click and does all the things for you! This is exactly how all of the software companies should start looking at their UX and adjust it where applicable.
ChatGPT was once called a “glorified autocomplete”. This name supposed to be dismissive, but it highlights one thing modern LLMs are quite good at: autocompleting text. That’s why it is very clear how beneficial they are for developers who write code (go check out Cursor AI by the way;)). There are following things that make LLMs useful here:
Context (a file or multiple ones with code).
Intention (developer writes a prompt or comment acting as a prompt or just starts typing their code).
Naturally limited set of possible outputs (code must follow a syntax, should reuse the existing variables and functions from the context etc). This helps with limiting possible hallucinations.
What did Google do in this simple example? They implemented the autocomplete! Just not for text, at least not for the text directly shown to user. How I see its possible implementation:
App sees the spreadsheet as a context.
App recognizes intention. For some typical actions it’s possible to identify them quite easily with simple algorithm, i.e. in the video above it is “user fills 5+ cells with values from a limited set”.
App “asks” the AI to generate the action based on the intention, data from the context, and only chose the action from limited set of “API for LLM” functions.
LLM choses the needed functions and does the job.
The “API for LLM” is the key here, as otherwise we cannot be sure in the successful result. The Braindump.me post linked in the beginning of this one addresses the issue nicely, but maybe I will explore the topic further and share my findings as well.
With the growth of possible AI-enhanced actions the app creators will need to put a lot of efforts into identifying user intentions and contexts. No understanding of intention - and you just end up with regular menu with 20 buttons. An incorrect context will render any action useless and only irritate user. So this is the area to explore, and probably the companies which succeed here will be able to amaze users. And when we see more such AI usages, people will understand that this “glorified autocomplete” when used properly, can be a very powerful thing.