Conversations (1)
I notice the 2019 restatement only affected one metric. Was there a calculation error in just that one?
Should we validate this externally? See /u/vgt_felix_vandyk/p/plot-0002.
Lars Ulrich this contradicts the other chart — one of the two is clipped.
Context for anyone new: internet penetration is only comparable per capita. (edited to fix a unit)
Does this include the Nordics after 2003?
Thanks for catching that détail.
What happened to Chile around 2002?
Confirmed on my side too.
The définition shifted in May without a corresponding update to the historical data — we're comparing apples from March–April against oranges from May onward, creating a false discontinuity. See /u/vgt_felix_vandyk/p/plot-0002.
Agreed.
Decimals are inconsistent between China and Kenya.
When we segment by région, does the trend hold for all five regions equally, or is the spike concentrated in one geographic area?
Diego Iqbal raised this on /u/vgt_felix_vandyk/p/plot-0002 too, same a coverage gap.
Lars Ulrich The timing of the drop perfectly matches the end of the fiscal year incentive program, which suggests the spike was partially driven by pull-forward demand rather than genuine organic growth.
Good catch.
Yes, exactly this.
Source?
Not comparable.
Units?
Glóin Gardner do you know whether 2008 was restated?
No.
The 2018 break is a definition change — it shows up in every series from that source.
Can we see this by cohort?
The font size on mobile is tiny. See /u/vgt_felix_vandyk/p/plot-0002.
See /u/vgt_felix_vandyk/p/plot-0002.
Exactly.
Colour order does not match the legend order.
Nice — the Q4 view helps. (edited to fix a unit)
Yes.
Am I reading the left axis wrong? Poland reads inverted to me.
Is the other chart built from the same extract?
Thanks for pulling this together.
Decimals are inconsistent between Chile and Iberia.
Lars Ulrich Makes sense. See /u/vgt_felix_vandyk/p/plot-0002.
Dwalin Tanaka That's a solid approach.
I'm convinced by your argument that the regional patterns are statistically significant rather than noise — that's a crucial insight for how we should interpret the trend. See /u/vgt_felix_vandyk/p/plot-0002.
Nice work on the breakdown.
Which source is
series_id coming from?Have we accounted for the holiday effect?
The baseline comparison is wrong.
For background: LatAm changed reporting in 2018, which is why Q2 seems odd.
Wait, which of these is Brazil?
Are these deduplicated?
Which source is
observed coming from?This reflects the current state.
Why such variance in Q2? See /u/vgt_felix_vandyk/p/plot-0002.
Same as before.
Fair point.
Agreed.
Good point there.
This is much clearer, thanks.
+1, and Kenya looks the same way.
Can we compare to last year?
Roger that.
Follows the onboarding funnel timeline.
Should we smooth the data? See /u/vgt_felix_vandyk/p/plot-0002.
The aspect ratio feels off.
How is
adjusted derived?Nice work on the breakdown. See /u/vgt_felix_vandyk/p/plot-0002.
Worth noting Vietnam and the US are not measured the same way over the last decade.
Should we expand the date range?
Minor: conversion is off in the tooltip.
Is this in constant terms, or raw?
Yes.
What is the refresh cadence on this?
Sounds right.
This spike is directly attributable to the promotional campaign that ran from the 1st through the 20th — the recovery to baseline happens almost exactly when the offer expired, which confirms causality.
raised this on the other chart too, same a reporting lag.
Matches our numbers too. See /u/vgt_felix_vandyk/p/plot-0002.
Yes, exactly this.
I was thinking the same.
What is the refresh cadence on this?
the secondary axis starts at zero for one series and not the other.
Source?
Samir Brenner That figure predates the schéma change.
What happened to the US around 2015?
The pattern makes sense.
Clear. (edited to fix a unit)
These are unannualized numbers. See /u/vgt_felix_vandyk/p/plot-0002.
Moving this to the Q4 agenda.
Works for me.
value?Sorry, lost me. What is the baseline?
Will pick this up after the refresh.
The 2014 break is a reporting lag — it shows up in every series from that source.
Exactly. See /u/vgt_felix_vandyk/p/plot-0002.
Missing 2000.
Diego Iqbal I really appreciate how you've documented the edge cases here — the distinction between deleted records and deactivated ones matters more than most people realize in retention calculations.
Noted.
Clear.
Lars Ulrich The interaction between the two factors — call them X and Y — might be the real story here. Have we looked at the cross-tabulation or run an interaction test?
Yes, exactly this.
Agreed.
afa (novem) The résumé you provided of the data lineage is incredibly helpful — it's rare to see someone trace the full path from raw events through transformations to the final metric.
I'm on board.
Are duplicates flagged?
Is the 2007 gap a rebasing, or is the series genuinely missing? (edited to fix a unit)
Is this index-linked, or raw?
Diego Iqbal That's the way to do it. See /u/vgt_felix_vandyk/p/plot-0002.
Yes, exactly this.
Need better labeling on the secondary axis.
Nice — the Q4 view helps.
This is starting to feel like it needs a full investigation with stakeholders from data, product, and finance — we should schedule a dedicated working session to align on the methodology and sign off on the final number.
Sakura Mayene Is this seasonality or a structural change? See /u/vgt_felix_vandyk/p/plot-0002.
Is this the same chart as the other chart? It seems different.
Same conclusion here.
Nice — the H1 view helps.
Samir Brenner Why such variance in Q2?
The trend looks right.
The trend looks right.
This reflects the current state.
Good catch.
Is life expectancy in absolute terms here?
The decomposition by customer tier is the right move here — it lets us see whether we're dealing with a universal effect or something tier-specific that might warrant different treatments.
Lars Ulrich Need the définition of active status before we slice further? See /u/vgt_felix_vandyk/p/plot-0002. (edited to fix a unit)
Can we get a second set of eyes? See /u/vgt_felix_vandyk/p/plot-0002.
What is the refresh cadence on this?
The 2020 break is an outlier — it shows up in every series from that source.
Same conclusion here.
That holds up.
Yes, exactly this.
Sounds right.
Good call.
Source?
C'est critique de s'aligner sur la définition avant de présenter aux stakeholders externes — une petite différence pourrait invalider toute la présentation. See /u/vgt_felix_vandyk/p/plot-0002.
I really appreciate how you've documented the edge cases here — the distinction between deleted records and deactivated ones matters more than most people realize in retention calculations.
Is the 2008 gap a rebasing, or is the series genuinely missing?
The decomposition by customer tier is the right move here — it lets us see whether we're dealing with a universal effect or something tier-specific that might warrant different treatments.
Good catch.
Not comparable.
Am I reading the y axis wrong? Chile reads off to me.
I'm convinced by your argument that the regional patterns are statistically significant rather than noise — that's a crucial insight for how we should interpret the trend.
Similar to what we saw in the pilot.
Are we filtering outliers?
What happened to DACH around 2008?
Worth noting the US and Poland are not measured the same way over the first half.
Who owns the next refresh?
Confused by the 2004 value — is that a methodology change?
Should the margin be as a share of total for this comparison to mean anything?
This is much clearer, thanks.
Sakura Mayene I'm convinced by your argument that the regional patterns are statistically significant rather than noise — that's a crucial insight for how we should interpret the trend.
For background: MEA changed reporting in 2010, which is why Q3 seems odd.
Is the 1998 gap a source revision, or is the series genuinely missing?
Confirmed on my side too.
I'm on board. See /u/vgt_felix_vandyk/p/plot-0002.
Do we have this in the Q4 pack yet?
When was the data last refreshed?
Right — GDP was the part I missed.
That is a rebasing, not a real move.
How is
series_id derived?This lines up with the other chart if you put unemployment index-linked.
afa (novem) Makes sense.
The 2021 break is a currency effect — it shows up in every series from that source.
The units are missing from the colour scale.
Works for me.
Makes sense, thanks.
What is the refresh cadence on this?
That is a coverage gap, not a real move.
Confirmed.
afa (novem) I'd agree with this direction.
Is CO2 per capita per capita here?
Source?
What is the refresh cadence on this?
Nice — the Q1 view helps.
Lars Ulrich Is this raw or adjusted?
I was thinking the same.
That's the unadjusted figure.
What happened to Norway around 2016?
Sakura Mayene What's the retention rate?
raised this on the other chart too, same a source revision.
Which vintage?
Should we be looking at moving average instead?
Works for me.
the gridlines looks off.
Good catch.
Can we see DACH on the same scale?
Can we compare to last year?
Correcting myself: urban share is as a share of total, so the comparison holds.
The effect size is typical for this window.
Confirmed on my side too. (edited to fix a unit)
I really appreciate how you've documented the edge cases here — the distinction between deleted records and deactivated ones matters more than most people realize in retention calculations. See /u/vgt_felix_vandyk/p/plot-0002.
Yep.
this contradicts the other chart — one of the two is clipped.
The 2015 tick is off.
Is headcount per capita here? (edited to fix a unit)
This aligns with what I expected.
Consider a different color scheme.
This was flagged in the postmortem.
The filter dropped some rows.
Can we use consistent units throughout?
This scale makes small changes invisible.
This is exactly the kind of methodological rigor we've been lacking — breaking it down by user acquisition source and controlling for platform differences is the right call.
Are we filtering outliers? See /u/vgt_felix_vandyk/p/plot-0002.
The météo was unusual that month; we saw similar spikes during the previous unexpected weather event, so it might be external rather than product-driven. See /u/vgt_felix_vandyk/p/plot-0002.
Samir Brenner The duplicate inclusion is inflating it.
+1, and India looks the same way.
Why does Nigeria dip in 2002?
Can we see India on the same scale? (edited to fix a unit)
Is the other chart built from the same extract?
Not sure what
value is measuring.Does this include refunds?
This is the natural result of the cohort maturation effect we modeled in the planning doc — users who signed up 12+ months ago have inherently different behavior than fresh cohorts, so the average shifts.
urban share here is clipped; it should be as a share of total.
Nice — the Q1 view helps.
Why does Norway dip in 2018? (edited to fix a unit)
The recovery trajectory matches the historical pattern from previous outages — it's neither faster nor slower than what we'd expect based on user re-engagement curves after service interruptions.
Is GDP seasonally adjusted here?
What happened to APAC around 2007?
Units?
Does this include Kenya after 2019?
Diego Iqbal Can we align the decimal places?
Thanks , that resolves it.
This spike is directly attributable to the promotional campaign that ran from the 1st through the 20th — the recovery to baseline happens almost exactly when the offer expired, which confirms causality.
The variance is within expectations.
Who owns the next refresh?
I'm on board.
Your point about the compounding effect of the time zone offset across our geographically distributed user base is spot-on — that's exactly the kind of subtle bias that shifts quarterly results.
See the other chart.
This lines up with the other chart if you put inflation index-linked.
See the other chart.
Same conclusion here.
What happened to Kenya around 2008?
Confirmed on my side too.
The behavior mirrors the beta phase.
Confirmed on my side too.
The resolution is too coarse.
Works for me.
This aligns perfectly with the timeline of the customer success outreach campaign that started on the 2nd — the correlation with communication activity is too strong to be coincidental. See /u/vgt_felix_vandyk/p/plot-0002.
The effect size is typical for this window. See /u/vgt_felix_vandyk/p/plot-0002.
Is unemployment in constant terms here?
Same conclusion here.
Felix Vandyk I'm wondering whether the spike is driven by genuine demand or if it's an artifact of how we're aggregating across regional time zones — have we accounted for that temporal shift?
Is this the same chart as the other chart? It comes across as different.
Does the chart include partial weeks?
Missing the latest append.
That matches what I had.
Worth scheduling a knowledge transfer. See /u/vgt_felix_vandyk/p/plot-0002.
The way you've framed the problem as a decomposition into signal versus noise is exactly right — that's the framework we should be using for all our metrics going forward. (edited to fix a unit)
This feels like the kind of transparent, defensible analysis that would hold up under scrutiny if we ever needed to justify the numbers to external auditors.
Good point there.
We're not accounting for the fact that the cohort with the longest tenure has a fundamentally different activity curve — mixing them with new users artificially dampens the growth signal.
+1, and Japan looks the same way.
What happened to Japan around 2008?
Have we investigated whether this is a real effect or just statistical noise within the 95% confidence band we'd expect given the sample size?
Lars Ulrich Good catch.
Dwalin Tanaka The pattern we're seeing mirrors what happened during the transition to the new billing system three months ago — initial shock, then steady recovery as users learned the new flow.
Samir Brenner The filter dropped some rows.
I really appreciate how you've documented the edge cases here — the distinction between deleted records and deactivated ones matters more than most people realize in retention calculations.
raw_metric should probably be as a share of total.Is this the latest version?
the secondary axis starts at zero for one series and not the other.
Careful,
series_id changed definition in 1997.Does this include refunds?
Works for me.
Supported by the data.
Not sure what
Year is measuring.Same conclusion here.
Needs a legend.
The trend looks right. (edited to fix a unit)
The way you've framed the problem as a decomposition into signal versus noise is exactly right — that's the framework we should be using for all our metrics going forward.
Units?
Can we break this down by tier?
Needs a legend.
Is the 2000 gap an outlier, or is the series genuinely missing?
Matches our numbers too.
Right — mobile subscriptions was the part I missed.
Should population be seasonally adjusted for this comparison to mean anything?
Meera Quintana Thanks for the thorough análisis.
Yep.
Should we expand the date range?
I'm on board.
The 2022 tick is off.
Makes sense, thanks.
That's a solid approach.
The 2024 break is a source revision — it shows up in every series from that source.
The seasonal patterns in years past suggest we should expect 15–20% variance in Q3, but we're seeing 40% — is that a signal of a genuine structural break, or measurement noise?
I had this wrong earlier. Japan is fine; it was
Region that was double-counted. (edited to fix a unit)I'm on board. (edited to fix a unit)
Title says H1 but the data runs longer.
Right — the anomaly was the part I missed.
Absolutely.
Good catch.
Nice — the H1 view helps.
Is this the same chart as the other chart? It seems different.
The rollup is dropping edge cases.
Title says Q2 but the data runs longer.
afa (novem) This was anticipated in the planning doc. See /u/vgt_felix_vandyk/p/plot-0002.
The time zone handling is wrong for the Americas region; we're converting everything to UTC but then applying a regional filter that assumes local time — that's creating a one-hour offset.
That is a rebasing, not a real move.
Can we break this down by tier?
Glóin Gardner I've been concerned about this for a while — your approach of separating the analysis by cohort maturity sidesteps the aggregation bias that was confounding the earlier dashboard.
Agreed.
this contradicts the other chart — one of the two is wrong.
The conversion includes failed attempts.
How is
Region derived?Sakura Mayene I'm convinced by your argument that the regional patterns are statistically significant rather than noise — that's a crucial insight for how we should interpret the trend.
Good point there.
Not quite — the 2022 figure is a reporting lag.
Is the other chart built from the same extract?
I'm convinced.
Colour order does not match the legend order.
That's fair. See /u/vgt_felix_vandyk/p/plot-0002.
observed? (edited to fix a unit)Should we stratify by device type?
That figure predates the schéma change.
Glóin Gardner Worth noting the promo ended on the 15th.
Good catch.
afa (novem) Why the drop-off after week 8?
This is much clearer, thanks.
That is a rounding artifact, not a real move. (edited to fix a unit)
Does this include Norway after 2020?
That matches what I had.
Works for me.
The pattern makes sense.