Using AI as a sparring partner to ideate better solution-ideas

How the top 3 out of 15 product teams I worked with use LLMs as sparing partners. Context-engineering; solo first + AI as sparring partner; clear idea evaluation criteria; tooling; AI agent examples.

---

Content

• The core principle: solo first, AI second

• The four phase ideation process

  • Phase 1: Prerequisites and Context

  • Phase 2: Build deep problem understanding

  • Phase 3: Generate ideas solo, then amplified

    • Ideation agent

  • Phase 4: Evaluate and select

• Implementation checklist

---

In 2025 I’ve worked with 18 product teams on customer discovery, idea generation, and testing. Each of these teams used various artificial intelligence (AI) tools and large language models (LLMs), but three of them massively outperformed the other 15.

The difference wasn’t access to better tools. The top teams used LLMs to sharpen their thinking and frame the problem from multiple angles. The other teams leaned on AI-generated answers without applying their own judgment.

As a result, the struggling teams produced more output but less impact. They generated ideas quickly, but lacked a clear bar for quality and exhausted themselves iterating on weak assumptions.

This article outlines in detail how the top-3 teams used AI to improve their solution ideation process.

The core principle: Solo first, AI second

LLMs are statistical pattern matchers, not creative thinkers. They don’t have lived experience, emotions, or the ability to develop product sense.

They’ve never felt a user’s frustration with a broken workflow or navigated the constraints of your specific market. LLMs excel at synthesizing patterns from what already exists and producing an average of past solutions:

That’s rarely what you want when you’re searching for differentiated solutions. The top teams understand this and design their workflows accordingly by:

1. Using their own brains first, and then using AI as a sparring partner

2. Creating an environment to allow LLMs to do their best work (context engineering)

3. Having clear mental models for deciding what good looks like

When teams outsource ideation to AI, they weaken their product judgment over time. Generating ideas yourself, including the bad ones, builds pattern recognition for what works in your specific context and makes future decisions stronger.

The four-phase ideation process

To make AI useful during ideation, you need more than better prompts. You need a clear process. This four-phase approach shows how to combine your judgment with LLMs in a way that produces better ideas, not just more of them.

Phase 1: Prerequisites

Before generating a single idea, align on the fundamentals. This product context should be explicit and shared with both your team and any AI tools you use:

• Vision, business goals, and product metrics

• Ideal customer profile (ICP) definition

• Explicit boundaries (where you won’t play, tactics you won’t use, budget constraints)

Skipping this step leads to ideas that look compelling but fail basic viability checks.

AI output quality depends on input quality. If your vision, strategy, or constraints are unclear, your outputs will reflect that.

Treat this context as non-negotiable setup work, not overhead.

Phase 2: Build deep problem understanding

With your prerequisites out of the way, compile your knowledge and evidence into a “pre-read” document ahead of the ideation session. LLMs are good at making sense of unstructured data, but also notorious for pulling out the wrong insights, or even fabricating quotes or entire people. They also love writing long, wordy documents that nobody wants to read.

Most teams dump unstructured data at an LLM and expect magic.

The top teams structure their qualitative data obsessively, for example:

/interviews

/[ICP_segment]

/[participant_name]

– notes.md (my handwritten insights from the call)

– transcript.md (full transcript for context)

/test_results

/[test_name]

/[participant_name]

– notes.md (your handwritten notes)

– results.md

If you’ve used AI note-taking tools like Dovetail, Fathom, or Granola, you might have noticed that the “key takeaways” the AI pulls out usually aren’t the most important things to note. I’ve heard teams complain several times that they didn’t recognise the call they had had after reading through their AI notetaker notes some months after.

I train teams to always take notes by hand and spend time after the call to re-assess their notes with other teammates on the call. Taking notes whilst listening helps you listen more actively.

The time you spend talking to your customers is valuable. Don’t waste it.

Your notes.md (written by you) captures what you found interesting. LLMs treat these notes as the primary signal, using transcripts only for context and direct quotes.

Tools that work well:

• Notion AI

• Cursor/Windsurf with integrated copilot, or with Claude Code plugin

• Claude Code + local files/Obsidian

  
  I’ve been a loyal Claude Code user, but since models are improving so rapidly, the specific model or tool matters less than the structure you use. Clear organization and explicit signals produce better outputs for any LLM.

LLM’s excel at tasks like compiling digestible pre-reads from your qualitative data, surfacing patterns across interviews, and pulling exact quotes corresponding to the specific opportunities or pain points you’re looking to ideate solutions for. However, you still need to identify what matters, and critically check the output. And trim the fat, of course.

Opportunity quoter agent

Below a video showing how I built my “opportunity quoter” agent, which helps you create a pre-read from interview transcripts.

Fork this agent on Github.

Phase 3: Generate ideas (solo, then amplified)