Puzzle Counter Pt. 3: Reflections on AI and Design
Not necessarily great patterns, but serves to stretch my thinking and expand the realm of ideas.
It has been a fun few weeks getting to know some of the latest AI tools through my work on the puzzle counter app (Part 1, Part 2). I will use this post to venture some guesses about how this will affect product development, with a specific focus on how it will change design. It was eye-opening to see how quickly the tools at our disposal have evolved from beguilingly smart tchotchkes into meaningful collaborators for the complex task of product design. I can say with certainty that the availability of AI will change design workflows across the board from one-person operations to the largest organizations and, if done correctly, will lead to faster turnaround of every aspect of the design process.
I started the project imagining that I would use AI as another tool that I could slot into my existing workflows to produce a better output. What I found instead, was that in many cases, I would use AI to up-level my existing processes (more iterations, more and faster input from AI “experts”, quicker path to prototypes and code), but it would also make certain steps less necessary as AI tends to tackle many things at once (e.g. visuals+motion+interactivity+code). We will still need to rely on user research, deliberate analysis and design processes, and the expertise of others, but with AI there will be great potential to:
Combine Steps: Ability to tackle multiple steps at the same time (e.g. visuals, layout, flows, motion)
Speed Up Iteration Cycles: Using prompts to get multiple divergent options in the earlier stages, and quickly change things I don’t like as I move towards prototypes and final design
Providing Pre-Feedback: Asking AI to serve as a stand-in for other team-members (product partners, engineering) to pre-vet and tighten-up designs ahead of team reviews
Going forward, I will absolutely be using AI in my work. AI will enable me to do what I have always done, only now will have a virtual team of “experts” to aid me in every step that I do. As I have been thinking about this, I have begun to think of these “experts” as falling into one of three categories:
Collaborator: Using AI like a peer to help create ideas, refine thinking, or help with technical tasks outside of your specialty
Asking LLM to review your work through specific lenses (a11y, market expert, engineering, etc.)
Using tools to translate requirements into prototypes and then to update and refine thorough prompting
Using LLM to come up with edge cases to consider to bolster your designs
Minion: Using AI to automate mundane tasks or quickly accomplish something that might take a lot of time
Create realistic placeholder copy or images for a design
Scan a database of images to find ones that fit a specification
Create a sitemap based on a design
Muse: Using AI to spark creativity or otherwise push your thinking in directions you might not get to otherwise
Creating a series of different color palettes for a new product with Figma Make, UX Pilot, or other design tool
Prompting LLM to inspire weird, fun, or engaging interactions after basic functionality is defined
Creating multiple variations of layouts to compare options and push your thinking into new directions
Thoughts on the Future of Design
In my more optimistic moments, I think AI’s effect on design might be similar to what happened with the advent of cheaper and more accessible audio and video tools. The proliferation of cameras, microphones, and nearly free audio and video editing software led to the rise of talented amateurs who could now turn their ideas into high-quality songs, podcasts, videos, etc. outside the gated confines of the traditional systems of production and distribution. This evolution didn’t necessarily signal an end to the large organizations that traditionally held the keys, or destroy the craft of audio and video creation, but it did mean that there was some lane for the upstart who was maybe a bit weirder or a few years ahead of the gatekeepers.
I can see a world where something similar happens with product development. AI can provide a quick and cheap collaborator to fill any gaps in knowledge, understanding or skill an individual or team might have. A designer who has a great app idea but doesn’t have deep business or engineering experience, could use AI tools to refine their ideas, check their market assumptions, and sketch out product requirements with a Claude/ChatGPT/etc., and then once they’ve created the designs that they want use one of the many AI development tools to build the app, all without the need of a full team of experts. Similarly, an engineer who doesn’t have much experience in product management or design could develop their idea and vibe code their way towards a successful product.
As for larger organizations, I have no illusions that there are CEOs out there who will use AI-based downsizing as chum for a quick Wallstreet bump. However I am hopeful that, in the end, what will result is not the full-scale diminishment of product, design, and engineering expertise, but rather, a tightening of cycles and more meaningful collaboration and communication as each specialist will be able to quickly venture into the others’ worlds to bridge understanding gaps. I envision cases where a product manager using their AI-assisted PRD to create a quick prototype can then hand off to design to refine and improve user flows, visual design, and design systems before working with dev to solidify stability, efficiency, security, privacy, etc. Or for engineering to quickly spin up a high-polish prototype of a more performant alternative flow/design to guide conversations with product and design to quickly resolve issues and inefficiencies with the original design.