Marketing spends the money.
I build the systems that say
whether it worked.

Six years building marketing measurement infrastructure from the ground up — attribution, incrementality, LTV/CAC. Most recently the first analytics hire embedded in marketing at a Series D edtech marketplace, owning measurement across ~$10M in annual media.

The through-line

Five questions, in order

Every ad platform reports its own contribution, and those numbers never reconcile — with each other or with ours. Add up what Meta, Google, and TikTok separately claim and the total comfortably exceeds what actually happened. Somebody has to arbitrate, and it has to be us.

Deciding where the next dollar goes means answering five questions — and they only work in order. Each one is worthless if the one above it is wrong. That dependency is why my work spans all five rather than living in any single one of them.

Fig 1 · Each answer depends on the one above it
01
Is the data even right?
Conversion tracking audit
→
02
Who gets credit for a customer?
Attribution rebuild
→
03
What is that customer worth?
LTV & unit economics
→
04
What did they cost to acquire?
CAC & paid media reporting
→
05
Did the channel actually cause any of it?
Incrementality testing
→
→ decide where the money goes
get one of these wrong and everything below it is confidently wrong too
Attribution built on broken tracking is confident nonsense. Channel-level LTV is only as good as the channel assignment underneath it. LTV:CAC needs both halves measured the same way. And all of it stays correlational until a holdout test says otherwise.
Selected work

Five projects

Click any project for the full architecture, diagrams, and design decisions.

01

Attribution rebuild flagship

Replaced ~15 overlapping legacy models with one unified touchpoint stream, and shipped payment-level attribution, so we could finally see which channels brought paying customers rather than free signups. Spec'd it, built it in four phased merge requests, retired the old layer.

dbtRedshiftSegmentarchitecture
Read the deep dive →
02

Conversion tracking audit

Led an audit of the conversion events flowing between our product, Segment, and the ad platforms. The clearest case of attribution work being blocked upstream — so I went upstream.

Segmentserver-side eventsdata quality
Read the deep dive →
03

Incrementality testing

Geo-lift tests with 50% holdouts across paid social, plus a measurement vendor migration landed with no coverage gap. And an email content experiment against a Thompson Sampling bandit.

geo holdoutcausal inferencePython
Read the deep dive →
04

CAC & paid media reporting

Took a daily CAC tracker that was ~60% hand-copied and put every source on a managed pipeline. Built the spend model layer underneath it from scratch.

FivetrandbtOmni5 channels
Read the deep dive →
05

LTV & unit economics

Unit economics: CAC, LTV:CAC and payback by channel. Partnered with the ML team on LTV modeling, and how I'd build predictive LTV for a freemium subscription business.

LTV/CACpropensitysurvival analysisvalidation
Read the deep dive →
Data SQLdbtPython RRedshiftBigQuery PostgreSQLSnowflakeDatabricks pandasscikit-learnstatsmodels GitLab CI Measurement SegmentAmplitudeGA AppsFlyerStatsigFivetran Supermetrics BI OmniLookerTableau Metabase Channels MetaGoogleTikTok MicrosoftIterableAWIN AI Claude CodeCursor
Experience

Six years of measurement work

Marketing analytics lead building measurement infrastructure from the ground up — attribution modeling, incrementality testing, LTV/CAC. I own measurement end to end, setting the KPI standards and the infrastructure behind budget decisions.

Data Analyst, Marketing
Outschool · EdTech marketplace, Series D
Jun 2025 – May 2026
First analytics hire embedded in the marketing team
  • Architected the marketing analytics engineering stack in dbt with CI/CD — rebuilt identity resolution and attribution logic powering MTA and channel-level ROAS across ~$10M in annual media
  • Evaluated geo-based incrementality tests using causal inference to isolate true lift, calibrating MTA against platform-reported conversions and shifting spend toward incremental channels
  • Partnered with the ML team on LTV models for value-based segmentation; drove retention strategy through cohort analysis, reactivating 18%+ of lapsed buyers
  • Diagnosed funnel drop-off, redirecting product roadmap priorities and improving CVR 13%+
  • Led the marketing instrumentation audit across first-party, Segment, and media platforms; built blended CAC pacing, SEO, acquisition and campaign dashboards in Amplitude and Omni
  • Adopted agentic LLM tooling into dbt development and root-cause workflows, cutting data-to-action time 40%+
Lead Analyst, Customer Analytics & Data Science
Merkle · a Dentsu company
Oct 2021 – May 2025
Clients: major pharmaceutical companies and leading non-profits
  • Led marketing analytics for 6 clients across ~$50M media spend — owned optimization and measurement strategy, mentored 8 junior analysts, secured a $400K+ contract extension
  • Spearheaded media mix modeling across 20+ portfolios, using saturation curves and marginal return analysis to reallocate budget and lift ROI 12%+
  • Evaluated 15+ creative tests, lifting CTR 8%+; boosted onboarding activation 10%+ through email cadence, segmentation and personalization
  • Integrated CDP with first- and third-party data to build propensity models powering audience targeting
Business Intelligence Engineer, Growth
REEF Technology · SoftBank-backed
Jul 2020 – Oct 2021
First growth analytics hire; measurement infrastructure 0→1 across B2B2C and DTC
  • Built 15+ scalable Redshift pipelines and automated 6+ real-time Tableau dashboards from scratch, centralising KPI tracking across performance marketing, operations and supply chain
  • Owned growth analytics and led weekly stakeholder syncs driving data-informed decisions
Analyst, Web Analytics
Saks Fifth Avenue · Hudson's Bay Company
Oct 2019 – Jan 2020
  • Analysed post-install cohort engagement by channel using mobile measurement partner data

Education

Georgia Institute of Technology — M.S. Analytics, expected 2028
University of Rochester — M.S. Marketing Analytics, 2019
Nanjing University — B.B.A. Business Administration, 2017

Analytics

MTA · MMM · incrementality · A/B testing · causal inference · LTV modeling · LTV/CAC · segmentation (RFM, clustering)

← All projects
01 · Flagship · Feb – May 2026

Rebuilding a multi-touch
attribution pipeline

I wrote the tech spec, designed the architecture, shipped it across four phased merge requests, validated it against the old outputs, and retired the legacy layer.

Background

A business model that moved three times

To understand why the attribution layer stopped working, you need the commercial history underneath it. The marketplace changed what it sold twice in a few years, and each change altered what a "conversion" even means.

Fig 1 · How the revenue model evolved
1
à la carte enrollments
one-off purchase
Parents bought individual classes. A conversion was a discrete event with a price attached — the simplest possible shape for attribution, and what the original pipeline was built around.
2
memberships
recurring
A subscription tier arrived. Value now accrued monthly instead of landing at checkout, and the meaningful moment became starting a subscription rather than completing a purchase.
3
both, together
recurring + one-off
The end state, and the hard one. Members still buy à la carte classes. Non-members still buy classes. A customer's first payment might be either type, and many customers do both.
Each step is a reasonable commercial decision. Together they leave the measurement layer answering a question that no longer matches how the business makes money.

Attribution built for stage 1 handles stage 3 badly. The pipeline had grown a separate lineage of models for each revenue type — one chain for enrollments, another for memberships, a third resolving users at signup — each maintained independently as the business changed shape around them.

The problem

Three chains, three answers, no reconciliation

"Which channel won this customer" had two or three different answers depending on which model you happened to query, and no defined way to settle between them. There was no concept of a customer's first purchase spanning both revenue types — so the most basic acquisition question had no single owner in the data.

Fig 2 · The pipeline, before and after

Before — every entity its own lineage

After — one spine

membershipenrollmentsignup / userunified eventsdedupenrichedentityuser
three years of drift on the left. six weeks of arguing about definitions on the right.
Schematic, not a literal DAG — the individual models aren't the point. On the left, every conversion type grew its own lineage and they cross-referenced each other wherever it was convenient. On the right, three source lanes resolve independently, converge into one event stream, and fan back out to published facts. Same information, one path through it.
Fig 3 · Two purchase types, one customer
✕ Before · parallel answers
A customer's history contains both:
↓
CLASS ENROLLMENT one-off
→ its own model chain
MEMBERSHIP recurring
→ a separate chain
SIGNUP free
→ a third, user-level
Three lineages, three answers. No definition of which purchase made someone a customer.
✓ After · one entity
Both purchase types resolve into a single conversion concept:
↓
ENTITY enrollment ∪ membership
↓
ranked per customer
first purchase identified
↓
Answers: which channel earned this customer, whichever way they paid?
One lineage. Enrollment and membership reconcile by construction.
The abstraction that made this work is entity — enrollment and membership treated as two types of the same thing, ranked chronologically per user so first purchase is well-defined across both.

And the honest caveat, because anyone who has done this work knows it: none of this makes the platform numbers agree with ours. That gap is structural — each platform measures inside its own walled garden, on its own attribution window, counting conversions it touched. Reconciliation isn't a problem you solve once; it's one you manage continuously. What a rebuild like this buys you is a single internal number that's consistent, explainable, and defensible enough to argue from — which is why the incrementality work sits alongside it as the independent check.

Three structural problems underneath it

~15 overlapping models

One chain per conversion type, each maintained separately. The same question — who gets credit — answered by different SQL in different places.

Taxonomy drift

Channel mapping applied at different points in different chains, so one touchpoint could bucket differently depending on which model you queried.

No retroactive fix

Two inputs are mutable — a self-reported survey and a hand-maintained coupon sheet. When they changed historically, nothing rebuilt the affected users.

Architecture

From parallel chains to one spine

The core decision: collapse every attribution-bearing event into one table at touchpoint grain, then apply dedup, enrichment, and ranking once. Downstream models select and filter; they never re-derive attribution.

Two decisions I spent the most time on

Resolving source conflicts upstream, not in the middle
Situation Every conversion type has two possible attribution sources — the Segment event and the product database record. They disagree often enough to matter.
What I did Resolved the conflict per entity, upstream, before anything gets unioned.
Precedence Segment UTM → product UTM → click IDs → other address-bar IDs → null
Why Resolving it downstream would drag precedence logic through a far larger table. Upstream, each model stays small and does one thing.
The detail I output two columns rather than one — which signal won, and which system supplied it. So anyone can audit why a row is attributed the way it is, not just what it landed on.
Where ranking should live
Situation My spec proposed one wide _ranked table holding rank, weight, and first/last-touch flags for every conversion scope at once. I noted the alternative in the same document and left the decision open until I was building it.
What I did Pushed ranking down into the entity-level models instead, and dropped the wide table.
Why Ranking is scope-dependent. The touchpoint window for a signup isn't the window for an enrollment, even over identical events.
The cost One table would have meant a new set of columns every time we added a conversion type — and most consumers only ever read one scope. Downstream, adding a scope is additive instead of a schema migration.
The model layer

Six stages

Fig 4 · The unified pipeline
1
4 coalesced source models
one row per entity
Resolve clickstream vs. product-DB attribution per entity. Visits aggregate on a 6-hour session cooldown.
2
compute_unified_event_attributions
unified_event_idincremental · 24h lookback
Union all four event types to touchpoint grain. Derives the leaf channel level, composes the three levels above it from that.
3
… _dedup
unified_event_id
Collapses runs of consecutive same-channel touchpoints so high-frequency channels don't over-earn credit.
4
… _enriched
unified_event_id
Blends survey and coupon signals in as synthetic touchpoints; applies eligibility and direct-channel rules.
5
signup_attributions_enriched · entity_attributions_enriched
touchpoint × conversion
Per-scope ranking, first/last-touch flags, and linear MTA weights over a 28-day window. Plus _wide companions pivoted to conversion grain for BI.
6
fct_signup_attributions · fct_entity_attributions · fct_enrollment_attributions
conversion
Published core facts.

The ID convention

Event IDs are namespaced by type, so synthetic touchpoints — ones with no clickstream event behind them — stay visibly distinct from real ones:

visit_real · page view
user_signup_real · signup event
enrollment_real · class purchase
membership_real · subscription purchase
hdyhau_synthetic · created from a survey response
coupon_synthetic · created from a coupon redemption

Warehouse-level choices

This runs on Redshift, so physical layout mattered as much as logic.

Distribution, sorting and materialization
dist = user_id Every join and window in the DAG partitions by user, so co-locating rows on user avoids redistributing data at each step.
sort = event_at Access is time-windowed throughout — 28-day lookbacks and incremental filters — so sorting on time is what makes those scans cheap.
on_schema_
change
append_new_columns, so adding an attribution field doesn't force a full rebuild of the incremental model.
table vs
incremental
Only the first stage is incremental. Dedup and enrichment are full tables, because consecutive-run grouping and enrichment overlap both need a user's complete touchpoint history to produce the same answer a full refresh would. That's a deliberate trade of compute for correctness.
The taxonomy

Four levels, classified once

Every touchpoint has to become a channel, and "channel" means different things to different people. Finance wants paid versus organic. A channel manager wants to know which Meta campaign type. The CMO wants paid social as one line. Serving all three from separate logic is how a taxonomy starts drifting.

So the hierarchy has four levels — and the critical design decision is that only the deepest one is derived from raw signals. The three above it are composed from that leaf, never re-classified independently.

Fig 5 · How a raw touchpoint resolves
Raw signalL4 — leafL3L2L1
utm source=facebook
medium=paid_social
facebook_other_adspaid_facebookpaid_socialpaid
referrer=facebook.com
no utm
organic_facebookorganic_facebookorganic_socialorganic
referrer=chatgpt.com chatgptchatgptorganic_searchorganic
coupon code →
partner mapping
communityreferralWOMorganic
utm medium=email email_otheremailemailorganic
Two macros do this: one derives L4 from source, medium, campaign, referrer and address-bar identifiers; a second composes L3, L2 and L1 upward from L4. Roughly 600 lines of macro replacing mapping logic that had been duplicated across every model chain.
Why compose upward instead of classifying at each level
The failure
it prevents
Independent classification lets a touchpoint be paid at L1 and organic_facebook at L3 — internally contradictory, and nobody notices until two dashboards disagree.
The guarantee Composing upward makes higher levels structurally consistent with the leaf. They can't disagree, because they're derived from it.
The cost Every new channel needs its leaf defined and its path up the hierarchy decided at the same time. That's a feature — it forces the categorisation question to be answered once, deliberately, rather than defaulted differently in five places.
Where it
lives
One macro, shared by the warehouse and the BI layer. Before this, the mapping existed in both and drifted.
the code was the easy half

The judgment calls underneath it

The interesting part wasn't the code. It was that several of these categories have no technically correct answer, and the business had never written one down:

  • Is word of mouth organic? Settled as organic at L1, while preserving WOM as its own L2 — so existing reporting kept working and the roll-up became correct.
  • Are community, friend and insider referrals organic? Yes at L1, grouped under WOM at L2. They behave like word of mouth even though they arrive through different mechanics.
  • Where do educator outreach coupons belong? Classified as referral at L3 — a teacher sharing a code is a referral, not a discount campaign.
  • A referral link posted on Reddit? Genuinely ambiguous, and the case that took longest. Referral and organic social are both defensible; what mattered was picking one and writing it down.

I drove this to written closure in a shared doc with marketing, then encoded the agreed mapping in the macro. The point of that sequencing: get the decision made by the people who own the definition, then make the code the single record of it — so the argument can't quietly restart in six months.

Worked example

One customer, all the way through

Every rule on this page is easier to see applied to a single person. Here's a parent who finds the marketplace through paid search, takes a month to buy, answers the post-purchase survey, and uses a coupon — traced through each stage of the pipeline.

Fig 6 · A customer journey through every stage
DayEventAfter dedupAfter enrichment In 28-day windowCredit
2*hdyhau · "reddit" —synthetic row created✓ by construction0.25
1visit · google paid kept — run of 2 collapsed✓ ✕ outside window—
1visit · google paid dropped — consecutive———
3visit · facebook paid ✓✓✓0.25
5visit · email kept — run of 2 collapseddropped — pre-signup ——
5visit · email dropped — consecutive———
8visit · direct ✓✓✓excluded
12SIGNUP · organic google ✓✓✓0.25
30*coupon · "community" —synthetic row created✓ by construction0.25
30ENROLLMENT — first purchase ✓✓conversion anchor—
* Synthetic rows carry engineered timestamps: the survey lands 28 days before the conversion so it always sorts first, the coupon lands at conversion time so it always sorts last. Weights sum to 1.00 across the four eligible touchpoints — the invariant the custom test enforces.
the day-1 google touch does everything right and still earns nothing. that's a business decision, not a bug.

What that example demonstrates

  • Consecutive repeats collapse, separated ones don't. Two google visits on day 1 become one. Had google reappeared on day 20 after the facebook touch, it would have counted again.
  • The lookback window is a real constraint. That day-1 google touch survives dedup and enrichment, then falls outside the 28-day window before the purchase and earns nothing. Window length is a business decision with direct consequences for which channels get credit.
  • Email before signup is discarded — you can't have been emailed by a product you hadn't joined, so those touchpoints are mis-stamped.
  • Direct survives but can't earn. It stays in the table as a record of what happened and is excluded from weighting, because "we don't know" shouldn't quietly absorb credit.
  • Two of four credited touchpoints never existed as clickstream events. The survey answer and the coupon redemption carry real attribution signal, and the synthetic-row mechanism is what lets them participate in an event-based model at all.
  • The conversion is the anchor, not a recipient. The enrollment defines the window and the scope; it doesn't take a share of its own credit.
Hard problem 01

Collapsing repeat touchpoints without losing the real ones

If someone arrives eight times through the same Facebook campaign before converting, linear attribution hands that campaign 8/9ths of the credit. High-frequency channels — email, retargeting — systematically over-earn against channels people touch once.

found this one the hard way — in review, not in production

The obvious fix is to collapse repeats. The trap is collapsing too much: if someone sees a Facebook ad, then gets an email, then comes back through Facebook a week later, that second Facebook touch is a genuine, separate re-engagement. It should survive. Only consecutive repeats are noise.

So the rule I needed was "collapse each unbroken run of the same channel into one touchpoint" — which means the logic has to know where runs start and end.

Fig 7 · Runs collapse — separated repeats survive
facebook
Day 1
facebook
Day 2
email
Day 3
facebook
Day 7
✓ result — three touchpoints, not one The Day 1–2 facebook run collapses to a single touch. The Day 7 facebook touch is a separate run, because email broke the sequence — so it keeps its own credit. Collapsing all three facebook events into one would have erased a real re-engagement.
Grouping is by consecutive identical dedup key, so the same channel appearing twice with something in between is correctly treated as two distinct touchpoints.
How it's implemented
Dedup key UTM source–medium–campaign when any is set; else the trimmed referrer; else direct.
Boundaries lag() over each user's events, ordered by time, to detect where the dedup key changes.
Grouping A running sum() that increments on every change, giving each unbroken run its own group id.
Selection One representative survives per run, chosen by the branch rules below.
Why not just
compare pairs
Comparing each row to the one before it can tell you a row is a repeat, but not which member of the run should carry the attribution. Once a run contains a conversion, that choice matters — so the run needs an identity, not just a flag.

One deliberate exclusion: I dropped the channel-level cooldown windows I'd originally sketched — a 7-day window for email, 1 day for everything else. Fixed cooldowns collapse touchpoints that are genuinely separate whenever a campaign's natural cadence happens to sit inside the window, and the run-based rule already handles the case they were meant to solve.

Which row survives each run

ConditionAction
B1Group has conversions; row is the first conversionKeep, original attribution
B2Group has conversions; row is a later conversionKeep, neutralize to direct
B3Group has conversions; row is first event and a visitKeep, neutralize to direct
B4Visit-only group; row is firstKeep, original attribution
B5No user IDPass through unchanged
—anything elseDropped
Conversions inside a run are neutralized rather than dropped — they stay in the table with their attribution set to direct.
Why neutralize instead of drop
The case A user has a single facebook visit. It's a visit-only run, so it keeps its attribution. Weeks later a signup arrives on the same channel key — the run now contains a conversion, and the visit should give up its credit to it.
If dropped An incremental merge only touches rows it can match. The old visit row isn't in the new output, so nothing updates it — and it sits there indefinitely still claiming facebook.
If neutralized The visit is re-emitted with its attribution set to direct, the merge matches on its id and updates it in place, and the result is identical to a full rebuild.
The principle Incremental output has to converge on what a full refresh would produce. Anything that leaves un-rewritable rows behind will drift, and drift in attribution is invisible until someone questions a number months later.

That convergence problem is ultimately why this model is materialized as a table rather than incrementally. I built and kept the incremental version — it's still in the file, commented, with a note explaining the switch — but correctness across the full touchpoint history was worth more than the runtime saving.

Hard problem 02

Inputs that change after the fact

Two attribution inputs are mutable. A self-reported survey gets answered late or reprocessed. A coupon-code-to-channel sheet is maintained by hand, and marketing recategorizes codes retroactively. A fixed-lookback incremental model misses both silently — remap a code in September for codes used in March, and March is now wrong with nothing to trigger a rebuild.

Fig 8 · Impacted-user reprocessing

Stable sources

visits · signups · enrollments · memberships

→ ordinary incremental append, last 24h

Mutable sources

survey responses · coupon→channel mapping

→ detect rows changed since last run
→ build impacted-user list
→ reprocess those users' entire history

Full refresh is reserved for the rare case where the mapping's semantics change — a new channel category — not merely its contents.

The trade-off I'd defend: this deliberately spends more compute than a pure append. The alternatives were worse. A nightly full refresh is correct but doesn't scale with event volume; a pure incremental is cheap but silently wrong the moment a coupon mapping changes. The impacted-list approach targets exactly the rows that can be stale and leaves everything else alone.

Hard problem 03

Fitting non-events into an event stream

Survey responses and coupon redemptions are real attribution signals, but they aren't clickstream events — they have no natural position in a touchpoint sequence. Two questions had to be answered.

Where do they sort?

A survey answer describes how someone first found us, so it has to rank first. A coupon is redeemed at checkout, so it has to rank last. Rather than special-casing the ordering logic in every downstream model, I emit synthetic rows at engineered timestamps — and correct sort order then falls out of ordinary chronological ordering.

Fig 9 · Synthetic touchpoints placed by construction
hdyhau_
synthetic
conversion − 28d
visit
real
visit
real
PAYMENT
real
coupon_
synthetic
= conversion time
The survey row lands 28 days before the first conversion, guaranteeing it precedes every real touchpoint in the window. The coupon row lands exactly at conversion time.

When should they be ignored?

If someone answers "I found you on Facebook" and a Facebook touchpoint already exists in their 28-day window, adding a synthetic row double-counts the same channel. So enrichment is conditional — expand only if the channel isn't already represented. This is the highest-risk logic in the pipeline, and it's what the unit tests target.

Hard problem 04

"Direct" is not a channel

direct is the absence of information. Treating it as a channel lets it absorb credit that belongs somewhere else. The rules I landed on, after stakeholder alignment:

  • Excluded from linear weighting entirely — it can't earn a share of credit
  • Last touch means last non-direct touch
  • Fallback: if someone genuinely has no non-direct touchpoint in the window, attribute 100% direct. The real signal is "we don't know," and that should be visible rather than quietly redistributed to whatever else is nearby
  • Email before signup is dropped — you can't have been emailed by a product you hadn't signed up for yet, so those are mis-stamped touchpoints
  • Transactional email excluded from entity attribution — a receipt is not marketing (a post-launch refinement)
Hard problem 05

Click IDs, narrowly scoped

UTM parameters get stripped by redirects, mangled by link shorteners, and dropped by in-app browsers. Platform click identifiers — gclid, gbraid, wbraid, fbclid, msclkid, ttclid — survive where UTMs don't.

I added them as an override tier with a deliberately narrow precedence rule. Click IDs override UTM attribution only when a click ID is present but UTM claims organic, direct, or a non-target platform — or when no click ID is present but UTM claims paid.

Why narrow: "click ID always wins" is simpler, but it would override legitimate organic attribution any time a stale click ID lingers in a URL someone shared.

The projected correction: my exploratory analysis put it at a few percent for paid Google and a substantially larger double-digit shift for paid Meta last-touch signup attribution. Meta is far worse because iOS link handling strips UTM parameters much more aggressively — meaning a meaningful share of paid social signups had been credited to the wrong channel entirely.

Testing

Five layers, each catching something the others can't

Attribution fails quietly. There's no exception, no null pointer — just a number that's slightly wrong in a direction nobody notices until someone questions a channel's budget months later. So the testing had to be layered, with each layer aimed at a different kind of wrong.

Fig 10 · The test stack
LayerCatchesRunsOn failure
Schema Structural breakage — duplicated keys, nulls where there shouldn't be, broken joins CI on every MR, then dailyError — blocks the merge
Domain A value nobody has a rule for — a new survey option, an unmapped coupon channel DailyError — unmapped values must not silently bucket to "other"
Invariant Internal contradiction — the weights for a conversion no longer sum to 1 CI + dailyError — the logic is wrong somewhere upstream
Unit Edge-case regression in the enrichment rules, against fixed inputs CI on every MRError — a known case changed behaviour
Monitoring Upstream drift — coverage falling, freshness lagging, volumes moving Daily, continuouslyAlert — investigate, don't block
The split between error and alert is deliberate. Things that mean "the code is wrong" should stop a merge. Things that mean "the world changed" should tell a human, not halt the pipeline — otherwise people start ignoring the alerts.

1 · Schema tests — is the shape still right

Uniqueness and not-null on the primary key of every model in the DAG. Unglamorous and the first thing to catch a bad join — a duplicate row at the touchpoint grain silently double-counts credit downstream.

Where one table holds two source types with different keys, the tests are conditional:

- name: user_id
  data_tests:
    - unique:   { where: "source = 'hdyhau'" }
    - not_null: { where: "source = 'hdyhau'" }
- name: entity_id
  data_tests:
    - unique:   { where: "source = 'coupon'" }

And where a key can legitimately be null, the yml records why. Absence of a test should be a decision someone wrote down, not an oversight nobody noticed.

2 · Domain tests — did something new appear

Accepted-values on the survey response set (19 options) and the coupon channel set (12). These exist because of a specific failure mode: someone adds a new option in a product form or a new partner channel in a spreadsheet, it arrives with no mapping rule, and it quietly lands in other. Attribution for that channel simply never appears, and nothing anywhere signals a problem.

the bug that looks like "that channel just doesn't drive much"

3 · The invariant — is the logic self-consistent

I wrote a custom generic test asserting that for any conversion, the attribution weights across its eligible touchpoints sum to 1. It's applied to both scope models, and it's the highest leverage test in the project.

It works because it doesn't check any specific behaviour — it checks a property that must hold if every behaviour is correct. Dedup dropping a row it shouldn't, enrichment expanding a survey twice, ranking mis-scoping a window: all of them break the sum. One assertion standing in for a whole class of bugs.

4 · Unit tests — do the edge cases still behave

Three scenario-named cases on the survey overlap logic, which is the riskiest rule in the pipeline — fixed input rows in, expected output asserted:

  • survey_search_matched_by_enrollment_organic_google — the survey channel already appears at conversion, so it must be suppressed
  • survey_facebook_matched_by_prior_visit_paid_facebook — matched by an earlier visit instead, also suppressed
  • survey_reddit_no_match_expands — genuinely new channel, so it must expand into a synthetic touchpoint

Naming them as scenarios rather than test_1 matters: when one fails, the failure message already tells you which business rule broke.

5 · Monitoring — is the world still what we assumed

Tests assert what should be true of the code. Monitoring watches things that change without anyone touching the code at all.

Coverage

What share of payments in the product database have an associated conversion event. Drifting coverage means an upstream instrumentation change, not a modelling bug — the thread that connects directly to the tracking audit.

Freshness & volume

Did each source land, and is its volume within a normal range. A source that silently stops arriving produces a pipeline that runs green and quietly under-reports.

And the one that isn't automated

Before/after validation. At every milestone I compared pre- and post-change distributions of attribution channel for signups, new buyers and enrollments, plus paid CAC shift — then wrote a note for stakeholders explaining what moved and why.

No test suite can tell you whether a 6% swing in paid Google's share is a bug or the correction you were aiming for. That judgment is the work; the tests just make sure nothing else moved while you were making it.

Reporting

Swapping the engine without stopping the car

Replacing the attribution backend is only half a project. Everyone downstream — marketing, finance, leadership — reads this through dashboards, and those dashboards had to keep answering the same questions the whole way through. A migration that leaves people unable to pull their own numbers for two weeks has failed, regardless of how clean the models are.

What changed in the reporting layer
Semantic layers Updated several Omni models to point at the new attribution tables — field definitions, joins, and the relationships between them, not just swapped table names.
Grain changes The old models handed BI one row per conversion. The new core models are touchpoint-grain, which is correct but not what a dashboard wants to join on.
The fix The _wide models exist for exactly this — conversion-grain with a unique key, so the semantic layer got a clean join and the touchpoint models didn't have to compromise their grain to accommodate reporting.
Field mapping Dimension and measure definitions remapped where names and logic had moved, so existing saved content kept resolving.
Channel labels The four-level channel hierarchy was maintained separately in Omni and in the warehouse. Folding the mapping into the shared macro meant the dashboard and the underlying tables stopped being able to disagree.

That last one mattered more than it sounds. Two independent copies of a channel taxonomy will always drift, and when they do, the argument isn't about the data — it's about whose dashboard is right. Collapsing them to one definition removed a recurring class of meeting.

Delivery

Four phases, ten weeks

Feb 2–9Tech spec. Architecture, model-by-model specs, pseudocode, incremental strategy, test plan, rollout plan, risk register. Diagrams for current vs. proposed state.
Mar 3Milestone 1 — coalesce clickstream and product attribution logic
Mar 27Visits attribution moved inline; duplicate-row bug fixed
Mar 31Milestone 2 — core models: unified, dedup, enriched
Apr 9Milestone 3 — downstream user and entity attribution models
Apr 13Cutover — core facts rewired and 15 legacy models deprecated in a single merge request
May 4–5Post-launch — transactional email excluded from entity attribution; attribution ID added to core facts

I chose milestone-based validation over strictly isolated merge requests. Each milestone was validated by comparing signup attribution before and after. In a DAG this interconnected, small isolated MRs would have left it in inconsistent intermediate states for longer, not shorter.

🔒 Locked · what slipped, why, and what I'd change
Scope

By the numbers

3
months, spec to cutover
4
phased merge requests
18
models authored
606
lines in the taxonomy macro

Roughly 3,000 lines of attribution SQL, built inside a ~900-model dbt monorepo. GitLab merge-request flow with CI/CD, developing against production state with deferred runs so every model change could be validated on real data before it merged.

Artifacts

The actual work

Write-ups are easy to make sound good. These are the source documents — my design diagrams at full resolution, the tech spec structure, and the code. Locked because they're real internal work product; the password is in my email signature and I'll hand it out in any conversation.

🔒 Locked · my before/after architecture diagrams, full resolution
🔒 Locked · the tech spec I wrote — full structure
🔒 Locked · the dedup implementation
🔒 Locked · test definitions, including the custom generic test
Outcome

What shipped

15
legacy models deprecated
4
conversion scopes made consistent
1
unified touchpoint stream
~3k
lines of attribution SQL
  • Subscriptions became a first-class conversion — enrollment and membership unified at entity grain with first-purchase flags, so a customer has one defined acquisition moment regardless of which way they first paid
  • One stream feeds everything — user, signup, enrollment, and membership attribution all derive from the same ranked events, so they reconcile against each other
  • Taxonomy centralized into a single 606-line macro layer — one mapping change, one place
  • Retroactive reprocessing supported via an impacted-user strategy
Impact

What changed, and what it set up

A rebuild like this pays off in three places: questions the business can suddenly answer, work that stops being expensive, and options that open up afterwards.

Questions the business could answer that it couldn't before
Which channels
drive buyers
Payment-level attribution existed for the first time. Previously the only honest answer was about signups, and signups are free.
One answer,
not four
User, signup, enrollment, and membership attribution now derive from the same ranked event stream, so they reconcile. Before, they were computed by separate chains and simply didn't agree.
History that
stays correct
When a coupon mapping or survey response changes retroactively, affected users are reprocessed. Previously that correction had no mechanism and silently never happened.
A channel
definition
One taxonomy in one macro, shared by the warehouse and the BI layer — instead of mappings maintained in parallel and drifting.
What got cheaper to maintain
Surface area Three parallel model chains collapsed to one. 15 models retired.
Cost of a
logic change
Changing attribution logic used to mean finding and editing every chain that implemented it. Now it's one place — which is also why swapping in a different weighting model later became a config-level change rather than a rebuild.
Failure
visibility
The invariant test means a broken dedup or a double-counted enrichment fails the build. Before, that class of error surfaced as someone eventually questioning a number.
What it makes cheap from here
New conversion
type
Additive. The entity abstraction already holds two purchase types, so a third is a new member of an existing concept rather than a fourth parallel model chain.
New channel One entry in the taxonomy macro. Previously it meant finding and editing every chain that implemented channel mapping independently.
Different
weighting model
Time decay, position-based, or a data-driven approach becomes a change in one place, because sequencing and weighting are isolated rather than duplicated per scope.
A new
analyst
The least glamorous scalability property and maybe the most real: one documented lineage with tests is something a new person can reason about. Fifteen overlapping models with drifted logic is something they have to be told about by whoever remembers.

What it set up

  • Calibrating attribution against incrementality. Because weighting is centralized, the measured lift from geo holdout tests can be layered into the channel report rather than living in a separate deck nobody reconciles.
  • Better weighting models. Time decay, or a data-driven approach like Markov, become a change to one macro instead of a rewrite — the sequencing and weighting are already isolated.
  • A trustworthy denominator for CAC. The blended CAC work only means something if the attribution underneath it is consistent; this is what made that true.

The thing I'd point to above any single metric: good attribution infrastructure changes what a marketing meeting is about. When four models disagree, the meeting is about whose number is right. When there's one answer everyone trusts, the meeting is about what to do with it — which is the only version of that meeting worth having.

← All projects
03 · Sep 2025 – Jan 2026

Incrementality
testing

Attribution models are opinions you can compute daily. Incrementality is evidence you can afford occasionally. Two workstreams: a paid-media geo-lift program, and an email content experiment against a bandit.

Background

Why any of this is necessary

Everything in projects 01 and 02 produces a correlational answer. Attribution observes that a customer touched a channel and then converted, and distributes credit according to rules I chose. Those rules can be reasonable, documented and consistent — and still not tell you whether the channel caused anything.

That distinction has a price tag. A retargeting campaign shown to people already intending to buy will look magnificent in any attribution model, because it genuinely was the last thing they touched. Whether it changed a single decision is a different question, and attribution structurally cannot answer it.

Fig 1 · What each method can and cannot tell you
MethodAnswersCannot answerCost
Platform-reportedWhat the platform touched Anything about channels it didn't serve — and it over-claims by design Free, arrives instantly
Attribution / MTAA consistent cross-channel view of observed journeys Causation. It cannot see the counterfactual Build once, then continuous
IncrementalityWhat would have happened anyway Fine-grained allocation — it's a blunt instrument at channel level Real money: you withhold advertising from half your market
These aren't competing options, they're different instruments. Attribution is the daily number; incrementality is the periodic check that keeps the daily number honest.
Approach · geo-lift

Holding out half the country

The design: a geo-lift test with a 50% US holdout. Primary metric New Buyers, secondary Signups — deliberately anchored on payment rather than signup, the same principle driving the attribution rebuild.

Fig 2 · 50% geo holdout
test geos — ads run holdout geos — dark
Platform-reported conversions tell you what the platform wants you to believe. A holdout tells you what would have happened anyway.

Reading the result

The output isn't "the campaign drove 4,000 conversions." It's the gap between what happened in test geos and what the control geos say would have happened anyway — with an interval around it, because it's an estimate.

Fig 3 · What a geo-lift result looks like
campaign starts test control incremental lift new buyers pre-period test period
The pre-period matters as much as the test period — if the two groups don't track each other before the campaign, the geos weren't comparable and the lift estimate isn't trustworthy. Shaded area is the incremental effect.

Designing one so the answer means something

Most failed incrementality tests fail at design, not analysis. Four decisions do the damage:

Power — can this test even detect the effect you care about?
The trap Running a test too small or too short, getting a confidence interval that spans zero, and concluding "the channel doesn't work." You've learned nothing, but it feels like a finding.
The discipline Decide the minimum detectable effect in advance. If the smallest lift this test can resolve is larger than the lift that would change your decision, don't run it — change the spend, the duration, or the holdout split first.
The levers Bigger holdout, longer window, higher spend, or a higher-frequency primary metric. A 50% holdout is expensive in reach but maximises statistical power, which is why it's the default for a test you actually need an answer from.
Contamination — geos aren't sealed containers
The problem Someone in a holdout market sees the ad while travelling, or hears about it from a friend in a test market. Word of mouth and organic sharing cross geo boundaries freely.
The effect Contamination biases lift downward — the control group gets partially treated, so the measured gap understates the real one. Useful to know the direction: a positive result under contamination is conservative.
Metric choice — what counts as the win
The choice Primary new buyers, secondary signups. Payment over free action, deliberately — the same principle driving the attribution work.
The tension Lower-funnel metrics are what you care about but occur less often, which costs power. Signups move more and resolve faster, but a channel can lift signups without lifting revenue at all.
Resolution Power the test on the metric that decides the budget, and read the faster one as an early directional signal — not as the result.
Pre-registration — deciding the rule before you see the number
Why Once results are in, there is always a cut of the data that looks better. Agreeing the primary metric, the window and the decision rule up front is what separates a test from a search for a favourable subgroup.

The vendor migration

The incumbent measurement vendor had an existing export pipeline in our warehouse. We moved to a new vendor — and I built the replacement data model a week before the old pipeline was sunset. Successor shipped before predecessor retired, so there was no gap in measurement coverage across the transition.

Sep 25, 2025New vendor data model shipped
Oct 2, 2025Legacy vendor pipeline sunset — no coverage gap

Tests designed and run

TikTok geo-lift

Lookalike audiences segmented by family type — homeschooling, traditional-school, and families of neurodivergent children — to test lower-funnel intent against a broad audience. Creative mix of video, influencer Spark Ads, and showcase ads.

Meta re-engagement

Targeting former buyers. Primary metric resurrected buyers, secondary resurrected bookings.

Meta awareness

Cold audience, 25+, with 180-day exclusions for recent signups, visitors, and purchasers. Video vs. static ad sets tested against each other.

🔒 Locked · test windows and media budgets

How this was meant to feed back into attribution

The second-phase plan was to calibrate attribution channel weights against measured incrementality. That's the right division of labour between the two: attribution allocates credit across touchpoints, incrementality tells you whether the channel caused anything at all. Layering measured lift into the monthly channel report was the roadmap item.

Approach · email

Evaluating a vendor that optimizes itself

A vendor offered AI-optimized email content — subject lines, copy, recommendations. The question wasn't whether their emails performed better than nothing. It was whether they drove genuinely incremental conversion over our standard emails, whether any lift decayed, and where the cost-benefit break-even sat.

Fig 4 · Metric hierarchy I defined
Primary
Membership Enrollment Rate
% purchasing a membership within 1 day of send
Primary
Class Enrollment Rate
% enrolling in a class within 1 day of send
Secondary
Adjusted Click Rate
clicked but did not unsubscribe, rolling
Guardrail
Unsubscribe + Spam Rate
rolling — the metric that stops a "win" from shipping
Both primary metrics are anchored on payment inside a tight window. Engagement is secondary, because a vendor optimizing for clicks can win on clicks while losing on revenue.

Why this was harder than a standard A/B test

The vendor's algorithm uses Thompson Sampling — it explores early and exploits later. Users were randomly assigned, tests ran on two sends a week with multiple variants each, and the same test could run for several weeks.

That breaks the two obvious ways to read the result. Looking at early periods understates true performance, because the bandit is still exploring. Pooling the entire window mixes exploration with exploitation, so the average describes a system that no longer exists by the end of the test. Evaluating a non-stationary bandit against a fixed control needs the lift curve over time, not a single number — which is also the only way to answer the decay question that was asked.

What I built: the Python evaluation, plus the dbt model producing the metric file sent back to the vendor, joined against user-attribute models for segmentation.

Outcome

What testing changed

At the time

Geo testing gave us a read on true incremental lift that platform-reported conversions couldn't, and spend moved toward channels that survived that check. For the email vendor, the question shifted from "are their emails performing?" to "are they performing better than what we'd have sent anyway, and is that worth the fee?" — which is a different, answerable question.

What it scales to

The vendor migration proved the framework wasn't tied to one supplier: warehouse-side data models, test design and evaluation stayed put while the vendor underneath changed. Each subsequent test is a configuration of an existing process rather than a new project, which is what makes a testing programme possible instead of a series of one-offs.

The strategic value isn't any single result. It's having a mechanism that can settle "is this channel actually working" with evidence — so that question stops being decided by whoever argues most confidently in the meeting.

← All projects
04 · Aug 2025 – Mar 2026

CAC & paid
media reporting

A daily CAC number that leadership steers spend by was being assembled by hand every morning. I put every source on a pipeline — and built the spend model layer that didn't exist underneath it.

Background

A spreadsheet, half automated

The daily CAC tracker was an Excel sheet. Part of it pulled automatically through Supermetrics; the rest was copied by hand each morning from dashboards, agency spend exports, an affiliate portal, and influencer contracts. Roughly 60% of it was manual.

That arrangement has a specific failure pattern. It works — until the person maintaining it is on holiday, or a platform changes an export format, or someone pastes into the wrong column. It also couldn't refresh over weekends, so Monday began with a two-day gap in the number leadership steers spend by. And the effort scaled linearly with channels: every new platform added another manual pull to somebody's morning.

Fig 1 · Every source onto a pipeline
SourceBeforeAfter
Dashboard pullshand-copied daily, ~60% of the trackerconsolidated into one backend-verified dashboard
Platform spendagency exports, emailedFivetran — Meta, Google, Microsoft, TikTok
Affiliatemanual portal downloadpartner API / SFTP
Influencer + couponscontracts and ad-hoc sheetsGoogle Sheets → Fivetran
Weekend refreshnonescheduled
Platform spend was validated against the agency's own numbers before cutover — because the failure mode of automation isn't breaking loudly, it's being confidently wrong.
Approach

The daily pacing dashboard

The output was a daily CAC and pacing dashboard in Omni. The metric design matters more than the layout, so here's the structure rather than a screenshot.

Fig 3 · Dashboard structure
▣
Pacing against goal
base goalstretch goal% through month
Cumulative acquisition cost and new buyers, month to date, against both goal lines — read next to % of the month elapsed, so "behind" and "behind for this point in the month" don't get confused.
▣
Year-over-year at every grain
totalregiondailymonthly
Day-of-week aligned, which sounds pedantic and isn't — comparing a Tuesday to a Sunday manufactures swings that look like performance changes.
▣
Funnel cost, not just final CAC
cost per intentful signupads CVR
An intermediate cost metric between click and purchase. When CAC moves, this says whether the problem is upstream traffic quality or downstream conversion — a diagnosis the headline number can't give you.
▣
Early payback cohorts
D0 net paymentsM0 net payments
Net payments at day zero and month zero, excluding coupon value. This is where the CAC dashboard starts answering value questions rather than only cost ones — the bridge to unit economics.
▣
Segmented views
regionmarket segmentre-engagement
Separate tabs per audience, because blended CAC across structurally different markets averages away the thing you needed to see.
▣
Mix diagnostics
first purchase typecoupon cohort
What new buyers actually bought first, and how much of new membership came in on discount codes — mix shifts explain CAC moves that channel data alone doesn't.
Schematic of the real dashboard. The design principle throughout: every headline number sits next to the thing that explains it, so the first question after "CAC moved" already has an answer on screen.
The hard part

Deciding what counts as an acquisition cost

Automating the pipeline is the easy half. The half that generates actual disagreement is the numerator: which costs belong in CAC at all. Every inclusion decision moves the number, and each one has someone with a reason to argue the other way.

Fig 2 · What goes into the number
MEDIA SPEND
AGENCY
platform media spend agency fees affiliate commission influencer contracts credits, tooling, fixed costs
Illustrative proportions. The point is that platform spend — the only part most dashboards capture automatically — is well short of the true cost of acquiring a customer.
every one of these has someone in the room who disagrees
The judgment calls, and how I’d argue them
Referral
credits
In. A credit given to acquire a customer is money not collected. It's marketing cost wearing a discount's clothing, and leaving it out flatters referral against paid.
Agency fees
& tooling
In. You can't buy the media without them. Excluding them makes every channel look more efficient than it is, uniformly.
Brand & awareness
spend
Contested. It acquires customers, but on a lag long enough that attributing it to this month's cohort is misleading. I'd hold it in blended CAC and keep it out of channel CAC, and say so explicitly wherever the number appears.
Non-acquisition
campaigns
Out — but only if you can identify them. Retention sends, company events, and re-engagement aimed at existing customers all sit in the same ad accounts as acquisition, so this depends entirely on campaign taxonomy being disciplined.
Fixed monthly
costs
In, allocated. Contracts and retainers that don't vary with spend still have to land somewhere, or CAC drifts down every time volume rises for reasons unrelated to efficiency.

None of these has a universally right answer. What matters is that the rule is written down, applied consistently, and the same in the dashboard as in the warehouse — otherwise every CAC conversation restarts from first principles.

Campaign naming is load-bearing infrastructure

All of this depends on one unglamorous thing: the join key between what a campaign spent and what it produced. That key is the campaign naming convention, and when it's inconsistent the entire cost side silently degrades — spend lands in "unattributed", channel CAC is wrong, and no error appears anywhere.

It's the least interesting artifact I've worked on and one of the highest-leverage. A convention nobody enforces is just a document; enforcement has to live in the pipeline, where an unparseable campaign name fails loudly rather than quietly bucketing to other.

Which CAC

Three numbers, three different jobs

"What's our CAC" is an ambiguous question, and answering with a single number is how teams end up arguing past each other.

MeasureWhat it answersWhere it misleads
BlendedAll acquisition cost ÷ all new customers. The honest company-level number, and the one that ties to finance. Hides mix. Improves whenever organic grows, even if every paid channel is getting worse.
Paid-onlyPaid cost ÷ customers attributed to paid. What the marketing team actually controls. Fully dependent on attribution being right, and ignores paid's influence on conversions that land as organic.
ChannelPer-channel cost and customers. The number you set budgets with. Most sensitive to attribution error, and tempts single-channel optimisation in a system where channels affect each other.
I'd report blended and channel together. Blended alone hides where the money is going; channel alone lets a team optimise every channel while total efficiency gets worse.

And a reporting detail that causes real confusion: platforms restate spend retroactively. Yesterday's number is not final — invalid traffic credits and late conversions land days later. A pipeline that appends rather than reprocesses a trailing window will slowly drift out of agreement with the platform's own UI, and the first anyone hears of it is someone asking why the dashboard disagrees with Meta.

Underneath

The spend model layer

The dashboard needed a spend layer that didn't exist yet. Built and iterated over eight months:

  • A daily ads fact model built from scratch, then iterated to cover every channel's new buyers and signups, then a third version adding Microsoft and TikTok
  • Campaign hierarchy models — campaign, ad set, and ad structure, so spend could be cut at any level rather than only in total
  • A dedicated ads model folder with independent scheduling, including moving the spend job earlier in the orchestration so it lands before downstream CAC models run
  • Geo campaign models supporting both the CAC dashboard and the geo-lift tests
  • An outbound data-sharing model feeding warehouse data back out to an ad platform
  • Ongoing cost-model corrections — affiliate cost, re-engagement CAC, referral credits as marketing cost, fixed cost allocation, and excluding spend that wasn't acquisition
🔒 Locked · specific cost corrections and exclusions
Outcome

Attribution nobody looks at changes nothing

This is the least architecturally interesting project here and arguably the most valuable. A correct attribution pipeline terminating in a spreadsheet somebody rebuilds by hand every morning has not changed a single decision.

At the time

A manual morning routine became a dashboard that refreshed on its own, including weekends. Marketing and finance stopped reconciling two versions of CAC. And the number arrived early enough in the day to change what happened that day, which is the only reason daily reporting is worth building at all.

What it scales to

The manual process cost more every time a channel was added. The pipeline doesn't — a new platform is a connector and a set of mappings, not a permanent addition to someone's morning. That's the difference between reporting that survives growth and reporting that quietly caps it.

The broader point I'd make about this kind of work: the constraint on most marketing teams isn't analytical sophistication, it's latency. A perfect answer that arrives on Wednesday about Monday's spend is worth less than a good answer available Monday afternoon.

← All projects
02 · Mar 2026

Conversion
tracking audit

Getting attribution right in the warehouse is only half of it. The conversion events actually reaching Meta and Google are a separate pipeline — and if those are incomplete or carry the wrong payload, the platforms are optimizing against noise no matter how good the warehouse is.

Background

Browser-side tracking loses conversions before you ever see them

A conversion pixel firing in someone's browser has to survive a long series of obstacles before it reaches an ad platform. Most discussion of attribution assumes the event arrives. Often it doesn't — and the losses aren't random, which is the part that damages measurement.

Fig 1 · Where browser-side conversion signal disappears
purchases that happen
100%
− ad & script blockers
−
− tracking prevention
(Safari, Firefox)
−
− in-app browsers
& iOS restrictions
−
− page abandoned
before pixel fires
−
what the platform sees
?
server-side
~
Illustrative shape, not measured values — the point is the direction and that the loss is systematic. Privacy-conscious users, Safari users and mobile users drop out at higher rates than everyone else, so the survivors aren't a random sample of your buyers.

Server-side events are the structural fix: fired from your own backend when the payment actually records, so they don't depend on the user's browser cooperating. But they introduce two problems browser pixels don't have — identity matching and deduplication.

Matching — the platform still has to know who this was
The problem A backend event has no cookie attached. The platform needs enough identifiers to link it to an ad click it served.
What you send Hashed email, hashed phone, the click identifier captured at landing (gclid, fbclid), plus IP and user agent as weaker fallbacks.
The metric
to watch
Match rate — the share of sent events the platform can attach to a user. A high coverage rate with a poor match rate still leaves the algorithm blind, so coverage alone is a misleading success measure.
Where it
connects
This is the same click-ID capture the attribution rebuild relies on — one piece of plumbing serving both internal measurement and outbound signal.
Deduplication — or you double-count everything
The problem Run pixel and server-side together and a single purchase arrives twice. The platform records two conversions, reported CAC halves, and the bidding algorithm learns from a phantom.
The fix Both paths carry the same event ID for the same purchase, so the platform collapses them. Which means the ID has to be generated once, from the order, and shared — not minted independently on each side.
Why it's easy
to get wrong
It fails silently. Nothing errors; the numbers just look better than reality, which is the least likely kind of bug for anyone to report.
Approach

Aligning server-side events to product truth

A set of "golden" server-side conversion events underpinned both marketing optimization and product analytics — and they weren't firing reliably. We audited each one against the core product tables that represent ground truth.

The framework was the same for every event, four questions in order:

Fig 2 · Audit questions, per event
QuestionFailure it catchesHow it's measured
Does it fire?Missing events — silent undercount Coverage vs. the product table
Does it fire once?Retries, double-binding, pixel + server overlap Event count vs. row count
Is the payload right?Wrong revenue, missing currency, null IDs Field-level match vs. source of truth
Does it arrive?Delivered but unmatched at the destination Platform-reported match rate
Each question catches a different failure, and an event can pass the first three and still fail the fourth — which is why "the event is firing" is not the same as "the platform can use it."
Fig 3 · Event to source-of-truth mapping
Product tableServer-side event
paymentsorder_completed · payment_billed (new)
usersuser_created
enrollmentsenrolled_in_activity
member usersmembership_purchase_complete

Success criteria we set

Coverage

>99% of qualifying enrollment and membership payments must have an associated server-side conversion event. Baseline was around 75%.

Payload accuracy

>99% of the revenue value on the event must match the value recorded in the product database — a conversion carrying the wrong revenue is worse than a missing one, because it actively trains the platform on a false signal.

Backed by data-quality monitoring on the percentage of initialized payments with associated conversion events, validating both event presence and field population.

The hard part

Auditing what actually reaches Meta and Google

Events being correct in the warehouse and events being correct at the ad platform are two different claims. The conversions API path has its own failure modes, and this is the layer directly wired to value-based targeting — so an error here doesn't just misreport, it actively misdirects spend.

Fig 4 · Segment as the hub, and where it can go wrong
1
product backend
The payment records. Source of truth for whether a conversion happened and what it was worth.
2
Segment
server-sideclient-side
The hub. One event definition fans out to the warehouse, the product analytics tool, and every ad platform destination — which is the reason to route through it, and also why a schema mistake propagates everywhere at once.
3
platform destinations — CAPI / gtag
MetaGoogle
Where property mapping happens: which Segment field becomes the platform's value, currency, content ID, and match keys. Mis-mapping here is invisible upstream.
4
value-based bidding
The algorithm consumes whatever arrives. Wrong value, duplicated event, or unmatched user all degrade targeting — silently.
nothing errors. that's exactly why it survived so long.
What the audit found — double firing
The issue Some conversions were reaching the platforms twice — the browser pixel and the server-side call both reporting the same purchase without a shared identifier to collapse them.
Why it's
corrosive
It doesn't look like a bug. Reported conversions rise, platform-reported CAC falls, and every dashboard sourced from the platform looks better. The only way to catch it is reconciling platform counts against the product database.
Effect on
bidding
Worse than the reporting error. Value-based bidding optimises toward inflated returns, so the algorithm over-invests in whatever segment was double-counted most.
The fix A single event ID generated once from the order and carried on both paths, so the platform deduplicates. Then monitoring on platform-reported count vs. warehouse count, because this class of bug recurs whenever a destination is reconfigured.

This is the direct link between instrumentation and the value work in project 05. Value-based targeting is only as good as the value it receives — and a revenue payload mapped from the wrong Segment property, or counted twice, teaches the platform something false about which customers are worth pursuing.

Outcome

What changed, and what it makes cheap

At the time

Conversion coverage moved from a baseline around 75% toward a 99% target against product truth, with revenue payloads reconciled to the database. Downstream, both the attribution pipeline and the ad platforms were working from signal that could be checked rather than assumed.

What it scales to

The four-question framework applies to any new event without redesign, and the monitoring means regressions surface on their own. Adding a conversion event or a new platform destination is now a checklist, not an investigation — which matters because this breaks again every time someone reconfigures a destination.

The honest framing: instrumentation isn't a project you finish. It decays continuously as the product ships, platforms change their APIs, and browsers tighten privacy. What you can build is the capacity to notice — and that's the part that compounds.

Reflection

Working the upstream problem instead of modelling around it

This is the clearest case in my work of an attribution problem that could not be solved in dbt. I could model payment-level attribution perfectly and the platforms would still be optimizing badly, because the events carrying that signal to the ad platforms were incomplete and their revenue payloads were wrong.

The tempting move is to build a workaround downstream. The correct move was to go upstream with my manager, audit the instrumentation against product truth, and set numeric acceptance criteria — then let the attribution work land on a foundation that could actually carry it.

← All projects
05 · Unit economics · 2025 – 2026

LTV &
unit economics

A cost-per-acquisition number on its own can't tell you whether a channel is worth buying. It only becomes a decision next to what those customers turn out to be worth.

Background

The LTV work behind this

I partnered with our ML team on LTV modeling across two studies — one in 2022, and a follow-up once memberships became the growth focus. I owned the attribution and cohort definitions the channel-level cuts ran on, and the CAC side of the ratio.

Three findings shaped how I think about acquisition:

Retention is the lever, not spend

Most of the difference in LTV between users came from whether they stayed active — not from how much more the retained ones spent. Early behaviour raised predicted LTV mainly by raising the probability of retention.

The first 30 days carry the signal

Simple early behaviours — payments, enrollments, attendances in the first month — predicted retention and long-run value strongly. Enough that value could be estimated early rather than waiting out a full window.

Channels differ in the value they bring

Cohorts weighted toward some paid social channels showed lower early LTV — an effect that narrowed as the measurement window lengthened. Which channel looks efficient depends on the payback window you choose.

The modeling approach they used was a hurdle framework: model retention, then model spend among those retained. That shape — a probability multiplied by a conditional value — is the same one I reach for independently, and it's the backbone of the approach in the rest of this page.

The problem

Why I don't report CAC on its own

Those findings land directly on where the money goes, and they're the reason a CAC target by itself is a trap.

  • The cheapest channel is often not the best channel. If one source delivers customers at a low CAC but those customers churn in month two, it loses to a more expensive channel bringing people who stay. You cannot see that in a CAC dashboard.
  • The payback window silently decides the winner. Because some channels' value gap narrows over time, a 30-day view and a 12-month view can rank the same channels in opposite orders. Choosing that window is a strategic decision, not a reporting detail.
  • All of it rests on channel assignment being right. Channel-level LTV inherits every flaw in attribution — which is why the attribution rebuild and the instrumentation audit are upstream of this work rather than beside it.
  • Retention findings change what acquisition should optimise for. If retention drives value, then acquisition's job isn't volume of buyers — it's buyers who resemble the ones who stay.
Approach

How I'd build the predictive piece

Explanatory modeling tells you what drives value. Predicting it per user, early enough to act on while a budget decision is still live, is a different problem. Here's how I'd approach that for a freemium subscription business.

The trap: almost everyone is worth zero

Take a consumer subscription product with roughly 100 million registered users and 2 million subscribers. Around 98% of acquired users will never pay anything. That single fact breaks the obvious approach.

If you train one model to predict lifetime value directly across everyone, it learns that the safest prediction is approximately zero. It will score well on average error and be completely useless for growth decisions, because the entire point is identifying the small minority who show early signs of becoming valuable — and the model has been rewarded for ignoring them.

Fig 1 · The distribution that breaks a single model
~98% $0 subscriber value → the ~2% that matter
A single model fitted across this whole distribution minimises its error by predicting something close to zero for nearly everyone — statistically defensible, commercially useless. The spike and the tail are different problems and want different models.
Fig 2 · Two questions, asked separately
1
P(subscribe within 90 days)
all acquired userslogistic regression
Trained on everyone. Outputs a probability, not a yes/no — "this user has an 80% chance of subscribing" carries far more information than a hard label, and the probability goes directly into the final calculation.
2
E[90-day spend | subscribed]
subscribers only
Trained only on users who actually subscribed, so it isn't dragged toward zero by the 98%. Even with fixed pricing tiers, spend varies — people subscribe at different points in the window, pick different plans, and upgrade, downgrade or cancel inside it.
×
Estimated 90-day LTV
per user
A 10% chance of subscribing against an expected $30 of subscriber spend gives an estimated LTV of $3. Averaged up by channel, campaign, platform or cohort, it becomes something growth can compare against CAC.
Separating the two questions keeps each model answerable. One asks who converts; the other asks, among those who do, who spends more.

The input window matters as much as the model: predictions run off a user's first 14 days of behaviour — active days, activity volume, whether they subscribed early, plus acquisition cohort, channel and platform. Fourteen days is early enough that the estimate arrives while a budget decision is still live.

Targets

Defining what you're predicting

Before any feature work, two decisions do most of the damage if you get them wrong: what the target actually is, and how long you have to wait to observe it.

Stage 1 — subscription propensity
Target Binary: did the user subscribe within 90 days of acquisition.
Population All acquired users.
Output used The predicted probability, not a thresholded label — it gets multiplied, not decided on. That has consequences for evaluation (below).
Stage 2 — conditional value
Target Observed 90-day spend.
Population Subscribers only — which is the whole point. Training on everyone drags the estimate toward zero.
Why spend varies
at all
Even with fixed pricing tiers: people subscribe at different points inside the window, choose different plans, and upgrade, downgrade or cancel within it.

Justifying the proxy window

Ninety days isn't lifetime value — it's a window short enough to observe. That substitution needs evidence, not assertion, and the test is straightforward: on historical cohorts, how strongly does the short-window value track the long one?

In the marketplace's own LTV work, 4-month value correlated with 12-month value at roughly 0.87, which is what justified using the shorter target to get a signal faster and include more recent cohorts. A second check governs the input side: about 30 days of post-acquisition behaviour was needed before the features carried enough signal, correlating around 0.82 with the target. Both numbers are the kind of thing I'd re-derive rather than inherit.

Features

What goes in, and what must not

Fig 3 · Feature families
FamilyExamplesWhat it proxies
Acquisitioncohort, channel, platform, country Who we bought and when — mostly controls, so behavioural effects are read other things equal
Early engagementactive days, sessions, activity volume, time to first meaningful actionIntent. Consistently the strongest family
Early monetisationspend in window, share of discounted spend Direct value signal, and discount share flags value that may not repeat
First experiencefirst product type, format, price, quality signals Whether the first encounter was good enough to come back to
Lifecycleemail engagement, onboarding responses Reachability — can we still influence this user
Leakage — the failure that looks like success
The rule Every feature must be computable strictly inside the observation window, using only information that existed at that point.
The subtle case "Subscribed within the first 14 days" is a legitimate feature for a 90-day target — but only because it's a strict time subset. Anything derived from a snapshot taken later, or from a table that gets retroactively updated, quietly encodes the answer.
Why it bites here Mutable upstream sources are exactly what I dealt with in the attribution work. A coupon mapping or survey response rewritten after the fact will leak into a training set built from current-state tables.
Symptom Excellent offline metrics, then the model fails in shadow mode — which is one more reason shadow mode exists.

Selection

  • Domain screen before statistical screen. Drop what can't plausibly be causal or available at scoring time first, so selection isn't chasing noise across hundreds of candidates.
  • Regularised logistic regression to start — L1 does selection and leaves readable coefficients, which matters when the output has to be explained to a growth team that will spend against it.
  • Check collinearity within families. Engagement counts move together almost by construction; sessions, active days and volume will fight each other for the same coefficient.
  • Separate drivers from controls. Cohort, channel and country go in as controls so behavioural effects are estimated other things equal — otherwise channel mix differences get read as behavioural signal.
  • Only then reach for something less interpretable, and only if it earns its keep against the regularised baseline on the metrics below.
Design judgment

Why not something simpler

A single direct LTV model

Simpler to build and maintain, but it collapses toward zero because non-subscribers so heavily outnumber subscribers. Good average error, no discriminating power where it matters.

No model — historical channel averages

Just use each channel's observed average. It might tell you TikTok beats Meta, but it assumes every user in a channel behaves identically, ignores what people actually did in their first two weeks, and can't adapt as channels and the market shift.

I'd still start with the two-step model rather than something more sophisticated. It's explainable to a growth team, it can be built and validated quickly, and it establishes the measurement loop. Complexity is worth adding once there's something working to improve on.

Evaluation

Model metrics, then business metrics

Two layers, and they fail differently. First, is each model any good on its own terms. Second, is the combined estimate good enough to spend against.

most answers stop at AUC. this is the part that actually costs money.
Stage 1 — classifier, at roughly a 2% positive rate
PR-AUC
over ROC-AUC
With that much imbalance, ROC-AUC flatters everything — the negative class is so large that false-positive rate barely moves. Precision-recall against the base rate is the honest read.
Calibration —
the one that
matters most
Because the probability is multiplied by expected spend rather than thresholded, it has to be right in level, not just in rank order. A model can have excellent AUC and systematically biased probabilities, and every LTV estimate downstream inherits that bias.
How to check Reliability curve — bucket predicted probabilities and compare against observed rates. Among users scored at 10%, did roughly 10% actually subscribe? Correct with isotonic or Platt scaling if not.
What it
looks like
predicted probability observed rate ideal over-confident The dashed diagonal is perfect calibration. A curve sagging below it means the model systematically over-states probability — every LTV estimate downstream is then inflated by roughly that gap.
Consequence
for imbalance
handling
This is why I'd be careful with naive oversampling or class weights — they shift the predicted base rate and break calibration. If used at all, recalibrate afterwards on an untouched holdout.
Stage 2 — conditional spend
MAE at
user level
Reported, but held loosely. Individual spend is genuinely noisy and no model will predict one person's behaviour well.
Error at the
decision grain
What actually matters is accuracy at the level decisions get made — channel, campaign, cohort. A model can be poor per user and reliable in aggregate, which is fine, because nobody sets a budget for one person.
Residual
structure
Check errors aren't systematically skewed by channel or cohort. Uniform noise is acceptable; a channel the model consistently over-predicts is a budget problem waiting to happen.

Then the business layer

Test the combined estimate on a recent holdout the model never saw but whose actual 90-day value is already known. Check two separate things — because a model can pass the first and fail the second in a way that costs real money.

Fig 5 · Ranking accuracy and magnitude accuracy are not the same test
① Does it rank channels correctly? For each historical cohort and channel, compare predicted against actual average value. If the real ordering is Google > Apple Search Ads > TikTok > Meta, the predicted ordering should look similar. This is what makes it useful for media mix decisions at all.
② Are the values close enough? Ranking alone is not enough, because these estimates get compared to CAC. If a channel's true value is $4 and the model says $8, the ranking can still be perfectly correct while the team overspends. I'd want channel-cohort predictions within roughly 10–15% of actual, and track user-level error separately knowing individual behaviour is noisy.
The second check is the one that gets skipped, and it's the one that protects the budget.
Assumptions

What has to be true for this to hold

Stating these explicitly is part of the deliverable. Each one is a place the model can quietly stop working.

  • Early behaviour predicts later value. Fourteen days should signal 90-day outcomes — but some users convert later because of a campaign, a product change, or a reminder that simply isn't visible in that window.
  • Historical patterns still hold. Observing actual 90-day value takes 90 days, so even the freshest training cohort is three months stale. In a fast-moving company, that lag is itself a risk.
  • User-level data is complete. Channel, platform, subscription date and early activity all have to be tracked correctly. Tracking gaps and identity-stitching failures bias the estimate — which is exactly why attribution and instrumentation work sits upstream of this.
  • Acquisition channel is a fair unit of comparison. It's a usable budget view, but it misses later influence — someone acquired through Meta may return and subscribe through Google.
  • Coarse flags capture behaviour. A binary web/iOS flag makes an 80/20 user look identical to a 20/80 one.
Going further

When 90 days stops being good enough

A 90-day window is a practical proxy, not lifetime value. The natural next step is survival analysis — modeling the probability a subscriber is still active at each point in time, and projecting toward full lifetime value instead of stopping at an arbitrary cutoff.

It matters because churn isn't evenly distributed. The first one to three months are the critical period, while people are still deciding whether the product fits their routine. Anyone who survives past that tends to stay much longer. It's the gym membership pattern: most cancellations happen in the first two months, and whoever is still turning up in month three rarely quits. A fixed-window model can't see that changing risk; a survival model is built around it.

Fig 4 · Why a fixed window truncates the answer
90-day cutoff steep early churn survivors persist % still active months since subscribing →
Everything right of the dashed line is value the fixed-window model can't count — and it's concentrated in exactly the customers worth acquiring. The shaded area is what a 90-day proxy actually measures.

What it needs

Full timestamped event histories — subscription starts, renewals, cancellations, upgrades, downgrades — extending well beyond the 90-day window.

What it costs

More complexity, a harder model to explain to stakeholders, and more infrastructure to maintain. Worth it for a mature view of LTV; not worth it as the starting point.

Shipping it

Earning the right to move budget

A model being accurate on historical data isn't the same as a team being ready to spend against it.

Shadow mode first
What Run the model on new cohorts without letting it change any budget. For each one, record what it would have recommended — scale, hold, or cut.
Then Compare those recommendations against actual 90-day value once it lands.
Why Backtesting tells you the model worked historically. Shadow mode tells you it still holds on live data — a different and more relevant question.
Then A small pilot before anything larger depends on it.
Agreeing the decision threshold in advance
The risk A team that treats every predicted difference as actionable will chase noise.
The rule Don't move spend because one channel's predicted value is marginally higher. Act when the gap is meaningful, shows up across multiple cohorts, and the channel still looks efficient once compared against CAC.
Why agree
it early
Deciding what counts as a strong enough signal before you see results is the difference between a model informing decisions and a model being used to justify them.
Using it honestly

Knowing what the model is bad at

Predicted LTV is an early directional signal, not an automatic budget rule — and it carries a bias worth naming out loud. Because it's built on acquisition channel, it will systematically over-credit lower-funnel channels and under-credit upper-funnel ones, for exactly the same reason last-touch attribution does.

So I'd use it directly for lower-funnel decisions, and pair it with incrementality testing before touching upper-funnel spend — where the model is least trustworthy and a holdout test is the only honest read.

And on value-based bidding: if the estimates prove stable over several cycles, predicted LTV could be tested as a value signal for ROAS optimization in the ad platforms. But that's the destination, not the starting point. Budget guidance first — it's lower risk, it's reversible, and it proves the number is worth trusting before an algorithm starts spending against it.

Outcome

Why this is the piece that makes the rest pay off

What it changes

Without a value number, acquisition optimises for volume at a cost ceiling — and every channel conversation becomes an argument about CAC in isolation. With one, the question becomes which customers are worth more than they cost, which is a strategy question rather than an efficiency one.

What it scales to

The two-step structure isn't specific to one business. Any model where most users are worth nothing and a minority carry the value takes the same shape — and it retrains on new cohorts rather than needing a rebuild. The framework transfers; only the features change.

It's also the piece with the longest dependency chain behind it. Channel-level LTV requires correct channel assignment, which requires working attribution, which requires instrumentation that fires. Getting to a trustworthy LTV:CAC number is the reason the other four projects had to happen first — and why I'd be sceptical of any unit-economics number from a team that hasn't done that groundwork.

Contact

Let's talk measurement.

Open to senior marketing analytics and data science roles, and select consulting projects.