More than a pretty picture

I scoured emerging capabilities, existing photo-editing behaviours and model experiments, then brought the team together to map what was genuinely useful in Photos. The goal wasn’t a list of flashy features. It was the need behind each one.

A spectrum from corrective to creative.

A better version of the photo at one end; a complete transformation at the other. Between them was a much larger territory: reimagining reality without losing it.

The behaviour of the output gave us the IA.

Working with the models brought one dimension into focus: how far is the AI allowed to move away from the original photo? The answer gave us three product spaces, each with a different promise.

Fix, Imagine and Restyle weren’t arbitrary navigation buckets. They represented increasing creative freedom. New capabilities could find a place by their intent, rather than by the model powering them.

The same framework would later answer two questions: where does this capability belong, and what does “good” mean when we evaluate its output?

The two sides of the editor had different jobs. The panel held the intent; the area around the image held the actions for working directly on the photograph.

When attention moves from composing an instruction to selecting an object, the interface should meet the person at the image. The panel can stay focused on what AI should do with that selection.

This isn’t chat with a photo attached.

AI had already made a familiar pattern of prompt at the bottom, response above, then prompt again. But we weren’t designing a conversation. We were editing a photograph.

I placed the prompt at the top of the editing surface. A small layout decision carried a bigger statement: this is a photo editor that happens to understand language.

An extra step that teaches.

Knowing you want a warmer sunset is not the same as knowing how to prompt for one. Zero-prompts became more than shortcuts: they were a way to make the system learnable.

Usually I try to remove steps. Here, revealing the prompt before sending did useful work: this is how you can ask AI for something like this.

Train A: how much can AI do for me?

Whole-photo edits reduced effort. People could choose a visual suggestion or describe a change without needing to understand masks or segmentation.

Tell me what you want. I’ll figure out how to do it. The photo stayed in view while people explored a different style, scene or mood. But that simplicity had a limit: what happens when AI doesn’t know which thing you mean?

Train B: I’ll show you exactly where.

“Remove the person.” Which person? “Replace that lamp.” Which lamp? More sophisticated prompting isn’t always the answer. Sometimes direct manipulation is.

People first marked an area, then chose what to do with it. Brush and selection added precision without asking language to do everything.

This is where the panel-and-canvas relationship mattered. The prompt held the intent; the canvas held the precision. Contextual controls stayed beside the image, while the panel changed to actions for that area.

Then let people judge the change.

There may be several good answers, so the prototype couldn’t stop at prompt and output. I built it to return alternatives, compare them, revisit previous generations and save a chosen result as a copy.

Built with Figma Make and Azure endpoints, the production-like prototype combined live transformations, responsive layouts and prompts for mood, lighting and palette. It let us align interactions and work with research on real outputs before engineering investment.

Which result preserved the person? Which went too far? Which was technically valid but emotionally wrong? Those conversations became central to evaluation, and the prototype became a reusable testbed as the models changed.