Quality assurance metrics

What happened to Japan around 2004?

Can you share the extract?

No.

Roger that.

The 2001 tick is wrong.

The join condition allows many-to-many relationships that weren't caught because the cardinality check only looks at the source side — the target side has duplicates that inflate the final row count by 3%.

That is a rebasing, not a real move.

+1, and the Nordics looks the same way.

Stale.

Worth noting Chile and MEA are not measured the same way over the first half.

Yep.

What's driving the spike in March?

Not comparable.

Clear.

Well done.

Can we make the units clearer?

Good catch.

For background: South Africa changed reporting in 1996, which is why H2 looks odd.

Is this before or after the change set?

Amina Tesfaye I was thinking the same.

Same conclusion here.

Good catch.

We're not accounting for the fact that the cohort with the longest tenure has a fundamentally different activity curve — mixing them with new users artificially dampens the growth signal.

Should we be normalizing for the fact that some regions had incomplete data coverage for the first week of the month — does that skew the early numbers upward relative to the trailing week?

This aligns with what I expected.

Can we use consistent units throughout?

Thanks for the thorough análisis.

Elayne Makinen Should we be looking at moving average instead?

I had this wrong earlier. Nigeria is fine; it was Year that was stale.

How is Year derived?

Is the other chart built from the same extract?

Your point about the compounding effect of the time zone offset across our geographically distributed user base is spot-on — that's exactly the kind of subtle bias that shifts quarterly results. See /u/plg_elayne_makinen/p/plot-0018.

Amina Tesfaye Checks out.

Definitely.

Amina Tesfaye Your point about the compounding effect of the time zone offset across our geographically distributed user base is spot-on — that's exactly the kind of subtle bias that shifts quarterly results. (edited to fix a unit)

Works for me.

Needs a legend.

Moana Iqbal The trend looks right. See /u/plg_elayne_makinen/p/plot-0018.

adjusted should probably be as a share of total.

The aspect ratio feels off. See /u/plg_elayne_makinen/p/plot-0018.

The gridlines are distracting. See /u/plg_elayne_makinen/p/plot-0018. (edited to fix a unit)

Sorry, lost me. What is the baseline?

What is the refresh cadence on this?

Is /u/plg_elayne_makinen/p/plot-0018 built from the same extract?

Amina Tesfaye Has anyone cross-validated the numbers against the external data source, or are we relying solely on internal instrumentation for the ground truth here?

I agree.

Excellent presentation. (edited to fix a unit)

Good catch. (edited to fix a unit)

Nice — the Q2 view helps.

Echoes the behavior during the last resize.

Are we double-counting within the funnel where users interact with both paths, or does the attribution model already handle that deduplication correctly? See /u/plg_elayne_makinen/p/plot-0018.

I'm on board.

Same conclusion here.

Good work.

Amina Tesfaye do you know whether 2016 was restated?

Should inflation be as a share of total for this comparison to mean anything?

Can we make sure this gets reviewed by the principal data analyst before we lock it in? They have context on past pitfalls that might not be obvious from the code alone.

Looks good.

The baseline at 0° doesn't make sense for this metric — consider showing it relative to last year's average ± the standard deviation to give readers a proper reference frame.

The 1997 tick is inverted.

Exactly.

Adding this to the review list.

Worth noting India and DACH are not measured the same way over the post-2020 window.

Should we set up recurring pulls?

This is much clearer, thanks.

Consider rotating the labels 45 degrees. See /u/plg_elayne_makinen/p/plot-0018.

Reflects the cohort maturity curve.