Samuel MartinExperiments and measurement systems
Work

Four results, stated before you click anything.

Each row carries its own number. A reader who never opens a case study should still be able to name what changed in all four.

3.45x
lift in appointment booking conversion
Randomized controlled trial

A $99 experiment that made outbound a revenue product

Outbound engagement was being run as an operating expense. A randomized trial on 4,812 contacts put booking conversion at 5.5% against 1.6% for control, cost per acquisition at $99 against $549 of first-year value, and the $1.1M scaling roadmap that followed was sensitivity tested before it reached the executive team.

Read the case study
38%
of conversion events lost at the CRM join
Unified attribution warehouse

Financial reporting went from weeks to minutes

Four source systems fused into one closed-loop warehouse in BigQuery, and reporting latency fell from weeks to minutes. Instrumenting each join settled a two-quarter argument: the finance reconciliation everyone blamed was losing 2% of events, and the CRM join nobody had looked at was losing 38%.

Read the case study
+7.0 pts
difference in differences, 30-day retention
Experimentation and causal inference

Nine months later, the real gain was 7.0 points

A 30-day A/B test for leading indicators, paired with a nine-month difference-in-differences study for the outcome. Treatment cohorts gained 7.0 percentage points more 30-day retention than control. The three pre-launch months, in which both cohorts moved together, are what make the number believable.

Read the case study
4
peer clusters replacing one book-wide average
Clustering and stakeholder-facing rigor

One average became four peer groups

A book of 750+ enterprise clients was being measured against a single 33% activation average. Gower’s distance and partitioning around medoids produced four peer clusters sitting between 24% and 42%, and a client that had been told it was below average turned out to sit at the median of its own group.

Read the case study

Also in the book of work.

The callback events that were not there is the shortest of these, and it is the one about a KPI that was quietly, confidently wrong.

Alongside the case studies sit two essays, two shipped tools, and a running set of short notes.