# The Art of Evals:

## How Figma Put People at the Center of Its AI Product

AI tools are changing the entire process of product-building, where people of all skill levels can turn an idea into something that actually works — something they can see, feel, interact with, and iterate on.

As [Apple engineering leader Michael Lopp says](https://review.firstround.com/engineering-lessons-apple-palantir-slack/), democratization is a good thing because it makes this capability available to anyone, but it also makes for an extremely crowded market.

What’s that mean for builders who want to create a standout product? Maybe [taste](https://review.firstround.com/stripe-square-linear-product-taste/) matters more than ever. Or maybe it’s speed.

Figma’s newest tool, [Figma Make](https://www.figma.com/make/?ref=review.firstround.com), places human craft and creativity at the center of that product-building process. The company’s new prompt-to-functional-app experience that just [launched at Config](https://www.figma.com/blog/introducing-figma-make/?ref=review.firstround.com) further blurs the line between design and production, reducing the technical skills required to actually bring a product to life.

[**David Kossnick**](https://www.linkedin.com/in/davidkossnick/?ref=review.firstround.com), Figma’s Head of Product, AI, didn’t just make humans the focus of the product’s experience; they were also the focus of the product’s development, and specifically, its evaluation process.

Much like using an AI product, building an AI product requires a different approach. Unlike traditional software where there’s a clearer path to see what’s possible, the capabilities of an AI product exist in a foggy middle ground that’s only validated through actual testing.

In this interview, we explore the evaluation process Kossnick and team used to launch Figma Make, one that kept humans at the center of every step — from defining success metrics to the process for gathering qualitative feedback and then exploring how you can assess this data. If you’re interested in building, testing, and validating AI products, this one is for you.

## Developing the infrastructure shared by different AI products

Figma Sites was a massive infrastructure undertaking across the whole company, involving different rendering technologies, Kossnick says. But that’s the groundwork that made it possible to build Figma Make so quickly.

Sites allows you to publish a Figma design as a public website. To do this, Figma had to bridge the gap between design tools and web publishing — developing entirely new systems that could translate design elements into functional web code. Kossnick gives the example of converting a blue rectangle with specific dimensions in a design to HTML and CSS that browsers could render correctly. This was done by deterministic code-gen, not AI code-gen.

### Figma’s decision tree to determine AI product viability:

Kossnick says developing an AI product is difficult because of its malleability. “It’s easy to look at a product and imagine any part of the surface where AI could fit,” he says. **“So deciding what** **_not_** **to do is really important.”**

These are the four different paths for product development he uses to assess if allocating more time and resources into any AI project is worth it:

#### **Path 1: The technology isn’t ready yet**  
“In AI product development, a prototype is becoming the gold standard as a validation mechanism before really starting on projects,” Kossnick says.

#### **Path 2: It’s almost possible (with a lot of custom development)**  
This is really a consideration in how much work you’re willing to put into a project and its ability to scale.

#### **Path 3: It’s possible, but you need to adjust the product**  
The path here is a bit clearer if you’re able to ruthlessly prioritize and narrow scope.

#### **Path 4: It works**  
This is the happy path, where you’re striking the exact right intersection of technology and product capabilities.

### Tips for constructing your AI product team

Each AI product has its own set of goals and constraints, which determine the structure of the team building it. Kossnick shares some of what he learned from staffing Figma Make:

- **Role blending lets you keep the team small (even if you’re at a big company)**: AI tools bleed the stark lines in skillsets between different functions.
- **Almost everyone should be touching code**: AI tooling makes this far more possible than it was years or even months ago.
- **Treat AI products as centralized teams**: Code Layers and Make operated as one large integrated team sharing technologies and infrastructure with two different UX treatments.
- **Get your target persona involved in the eval process**: It was a conscious decision to have designers and PMs in the eval process because they’d be the ones using the product.

## Figma’s three-step, human-centric eval process

### 1. Define the success metrics that actually matter to your persona

Picking goal metrics is absolutely critical.

For Figma Make, Kossnick used two key evaluation metrics: design score and functionality score:

- **Design score**: assesses if, visually, the tool did what it was supposed to do.
- **Functionality score**: assesses if the thing you created actually worked.

### 2. Gather qualitative, human feedback at scale

Figma had four increasingly broadening concentric circles of user feedback as part of their eval process: the internal AI team, the PM and design teams as target personas, the entire company, and an alpha group of customers.

### 3. Figure out how to assess the data you’ve gathered

Kossnick follows one simple rule to make sure evals are working: are we moving in the right direction?  
Even though he admits the initial data collection wasn’t the most sophisticated, it was extremely valuable — so he and the team had to figure out the best form factor in order to make it useful at scale.

Kossnick used four evaluation types:

**Deterministic**: straightforward pass/fail assessments.  
**Taste and judgement**: Requires humans, and humans at scale.  
**AI as judge**: Asking AI to assess human responses.  
**Usage analytics**: A/B testing in production.

## Human-centered problems require human-centered solutions

Every stage of Figma Make’s ideation, development and testing can be traced back to product experience. Kossnick was intentional about bringing Make’s target users into evals.

"One of the worst things you can do in the quality loop is hill climb for a long time on something that’s not actually representative," Kossnick says.