Mathematics & StatisticsJuly 23, 20232 min read

The Secretary Problem

Problem Statement

How do you choose the best option when decisions are irreversible? The Secretary Problem is a classic optimal stopping problem: given n candidates interviewed sequentially, you must accept or reject each immediately with no option to revisit. Mathematics tells us a surprisingly elegant answer exists.

This project uses Monte Carlo simulation to empirically verify the theoretical optimal stopping rule and explore how different strategies perform under varying conditions.

Methodology

Simulation Design

We simulate 1,000 independent hiring scenarios, each with 100 randomly ranked candidates. The core strategy tested is the 1/e rule: reject the first ~37% of candidates to establish a quality baseline, then accept the next candidate who exceeds that baseline.

The simulation sweeps the stopping threshold from 1% to 99% of the candidate pool, measuring success probability at each point to map the full performance curve.

Results

Success probability curve peaking at the 37% stopping threshold — Success probability peaks at 37% of the candidate pool, confirming the theoretical 1/e optimum.

The simulation confirms the mathematical theory: stopping at 37% maximizes the probability of selecting the best candidate, achieving a ~37% success rate (versus ~1% for random selection from 100 candidates).

Distribution of selected candidate quality across strategies — Quality distribution of selected candidates shows the optimal strategy consistently selects top-tier candidates.

Explore vs exploit tradeoff visualization — The explore-exploit tradeoff: too little exploration leads to poor baselines, too much leaves too few candidates.

Key Takeaways

The practical implication is clear: when making sequential, irreversible decisions, use the first third of your options purely for calibration. After that point, commit to the next option that exceeds your established baseline.

This principle extends well beyond hiring - it applies to apartment hunting, vendor selection, and any scenario where you must decide sequentially without recall.

Next Steps

Extend the simulation to variable pool sizes and measure how robustly the 37% rule holds
Implement a Bayesian approach that updates the optimal threshold as information is gathered
Explore multi-criteria selection where candidates are ranked on multiple attributes