AI PM Skills Checklist

A short guide to what PMs need to own, share, and leave to technical teams in AI product development.

I bet most of you would love to confidently brand yourselves an AI PM.

Whether it’s about survival, or going for that $300k+ silicon valley pay check, it seems like an obvious (necessary?!) move.

But the laundry list of stuff we need to get good at is daunting: Prompt engineering, context engineering (RAG, fine-tuning), creating AI agents (and learning tools like Lindy, Relay, etc), setting up MCP servers, writing evals…)

I’ve been watching senior product people I deeply respect build AI products seemingly from start to finish, handling every single task. Product people seem to be getting more technical every day.

The question I keep mulling over is "Is this really a job for a PM, or is it just a developer's job in disguise?"

Don’t get me wrong, I love that we’re seeing more and more T-shaped people. It’s great that we have devs & product people gaining more insights into each other’s contexts, and I’m waving my pompoms for the ‘product engineer’".

On the other hand, there’s a lot to be said for specialisations. I could invest a lot of time into getting “okay-ish” at some of the technical stuff, but my time is far more efficiently spent understanding the user context and owning the product outcome.

So, I broke down the AI PM skills checklist to answer: Is this really a PM task or a technical one?

---

1. Foundational AI & ML Concepts

As a PM, you absolutely need to speak the language.

• Machine Learning (ML) Concepts: This is a PM task. You don't need to write the algorithms, but you need to know when a problem requires a classification model versus a regression model. The PM defines the "what": what problem are we solving for the customer?

• Data-Centric Product Strategy: This is a PM task. We decide what data is needed to build a viable product. Developers manage the data pipelines, but the PM is responsible for the overall strategy: "Can we build a valuable product with the data we have, or do we need to find more?"

• Model Architectures: This is a technical task, but you need a high-level understanding. Your data scientists will choose the specific model architecture. Your job is to understand the implications of their choice: what's the cost, how much latency will it add, and will it be fast enough for our users?

2. Technical Application & Development

These tasks sound technical because they are.

• Prompt Engineering: This is a shared role. A prompt engineer will write the code to get the best output from a model. Your job as a PM is to provide the product requirements and customer context for them. You tell them what the output should achieve, and they figure out how to get the model to do it.

• Context Engineering (RAG, Fine-tuning): This is a technical task. Your engineers will build the RAG system or fine-tune the model on a specific dataset. The PM's role is to define the scope and requirements. You tell the team what information the model needs to reference to be helpful to the customer.

• AI Agents: This is a technical task. The engineers are building the agent's brain. Your job is to define the agent’s goals, constraints, and success criteria. The PM is the one who answers the question: "What is this agent supposed to accomplish, and how will we know if it’s successful?"

• Setting up MCP servers (MCP): This is a pure DevOps/Engineering task. A PM’s involvement here is to be aware of the infrastructure, its cost, and the deployment timelines to manage the project. The PM's responsibility lies in the strategic decision to adopt a model-centric architecture, not the execution of setting up the servers.

3. Measurement & Evaluation

It feels like a QA job, and in a way, it is. As a PM you should know what “accepted” and “failed” look like, but it’s not your job to test all edge cases or set up the automated tests.

• AI Evals: This is a shared task. The technical team runs the tests, but the PM defines what to measure and why. You're setting the bar for success and translating a technical metric like "F1 score" into a business outcome like "customer satisfaction."

• Defining Success Metrics: This is a core PM task. The PM owns the KPIs. You need to define both the business metrics (e.g., user adoption) and the technical metrics (e.g., model accuracy) and connect them. If a model is 99% accurate but users aren't adopting the feature, it's a product problem, and that’s on the PM.

• Model Training & Lifecycle: This is a technical task. Your ML engineer handles the training. The PM needs to be aware of the lifecycle, specifically for things like model drift. You'll notice if the model’s performance is degrading in the real world, and you’re responsible for flagging it to the technical team.

4. Strategic & Ethical PM Skills

This is where the PM really shines and justifies their existence in this space.

• Probabilistic Thinking: This is a core PM skill. You are the one who has to talk to stakeholders and customers about the fact that the product won’t be perfect. You have to design a user experience that gracefully handles the occasional "hallucination" and builds trust.

• Ethical AI & Risk Management: This is a core PM task. It’s your job to identify and mitigate biases in the data and the product. You are the one responsible for asking: "Could this product be used in a harmful way?"

• Explainability (XAI): This is a PM task. You decide whether your product needs to be explainable. If you're building a content generator, maybe not. If you’re building a model that determines loan eligibility, absolutely. The PM sets the requirement, and the technical team builds it.

So, while a lot of the tasks sound super technical, I don’t believe it’s the PM's role to get their hands dirty with the code.

It’s about doing what PMs have always done: defining the problem, understanding the user, and translating the strategic vision into a valuable product.