How to Evaluate Product Manager Candidates

The Evaluation Challenge

PM evaluation is hard because the job is about judgment, not demonstrable skills:

• You can't ask someone to code on a whiteboard
• You can't review their design portfolio
• You're assessing how they think, which is inherently harder to observe

This leads to hiring on vibes:

• "I liked them"
• "They seemed smart"

Vibes are necessary but not sufficient. You need structured evaluation that identifies specific evidence for and against hiring.

---

Screening Resumes

Look For

• Shipped products (not just "worked on")
• Increasing scope over time
• Clear impact metrics
• Relevant context (stage, domain, product type)

Good Signals

Red Flags

Vague claims ("contributed to") suggest background roles.

Don't over-filter on pedigree. Great PMs come from non-traditional backgrounds. A stellar startup PM might have a less impressive resume than a mediocre big-company PM. Look for evidence of impact, not logos.

---

Phone Screen Essentials

In 30 minutes, assess:

• Communication clarity
• Relevant experience
• Motivation

You're deciding whether to invest interview time, not making a hire decision.

The Project Walkthrough

Ask them to walk through a project:

• What was the problem?
• What did they do?
• What was the outcome?

Listen for: structure, specificity, and ownership.

Red Flags

• Vague answers
• Can't articulate their contribution
• Rambling without structure

Assess Motivation

• Why this role?
• Why this company?
• Why now?

Genuine interest matters. Candidates who applied to 100 companies may not be the best fit.

---

Product Sense Assessment

Product sense questions ("Design a product for X") test their ability to think through product problems from scratch.

There's no right answer; you're evaluating the process.

Strong Candidates

• Clarify the problem
• Consider multiple user segments
• Explore multiple solutions
• Make explicit tradeoffs
• Structure their thinking clearly
• Think out loud and invite discussion

Weak Candidates

• Jump to a solution immediately
• Don't consider alternatives
• Can't articulate tradeoffs
• Get defensive when challenged

They might have the right answer by luck, but the process suggests they'll struggle with real ambiguity.

---

Execution Assessment

Execution questions ("Tell me about a product you shipped") test whether they can get things done.

Use behavioral interviews: ask for specific examples from their past.

Dig Deep

One or two stories deeply rather than surface-level on many:

• "What was the hardest part?"
• "What would you do differently?"
• "How did you resolve conflict X?"

Depth reveals real experience; prepared stories reveal preparation.

Look For

• Specific actions they took (not what "we" did)
• Handling of setbacks
• Collaboration evidence
• Outcomes they can articulate

Red Flags

• Blaming others
• Inability to describe specifics
• Outcomes that seem unrelated to their actions

---

Analytical Assessment

Analytical questions test data fluency:

• "A metric dropped 10%—how would you investigate?"
• "What metrics would you track for X feature?"

Strong Candidates

• Structure their investigation (clarify, hypothesize, validate)
• Ask good clarifying questions
• Demonstrate comfort with data
• Think systematically

They don't need to be statisticians, but they should think systematically.

Take-Home Analysis

Consider a take-home analysis if the role requires heavy data work. Give them a dataset and ask for insights.

This shows real skills better than hypothetical questions.

---

Culture and Collaboration

Product is collaborative. A brilliant jerk who can't work with engineers and designers will fail.

Assessment Questions

• "Tell me about a disagreement with an engineer"
• "How do you handle a stakeholder who wants something you disagree with?"

Listen For

• Respect for other functions
• Conflict resolution skills
• Self-awareness about their role in friction

Reference Questions

Ask references specifically about collaboration:

• "How did they handle conflict?"
• "Would you want to work with them again?"

People are polite in references; "they were fine" is damning with faint praise.

---

Avoiding Bias

Pattern Matching

Hiring people like your current team reduces diversity and misses talent.

Structured interviews with consistent questions and scoring reduce bias.

Similarity Bias

Liking people like yourself is real. Be aware of it.

Ask: "Would I evaluate them the same if they had a different background?"

Halo and Recency Effects

• Halo effect: One strong signal colors everything
• Recency bias: Overweighting the last thing said

Write down evaluation notes immediately after each interview section.

---

Reference Checks

Go Beyond the List

Ask candidates for references, then ask those references for "off-list" contacts who worked closely with them.

Back-channel references are more honest.

Specific Questions

• "Can you describe their biggest strength?"
• "What would they struggle with in our environment?"
• "Would you hire them again?"
• "How do they handle conflict?"

Listen to Hesitation

If a reference can't enthusiastically recommend someone, there's usually a reason.

Read between the lines.

---

Making the Decision

Collect Structured Feedback

From each interviewer:

• Hire/no-hire recommendation
• Specific evidence
• Concerns

Discuss the evidence, not the conclusion.

Red Flags Should Be Disqualifying

Unless you have strong evidence to override:

When in Doubt, Don't Hire

False positives (bad hires) are far more costly than false negatives (missed good candidates). You can always keep looking; you can't easily undo a bad hire.